[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:48:07 dist-test-slave-cbsf krb5kdc[19113](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:48:07 dist-test-slave-cbsf krb5kdc[19113](info): set up 2 sockets
Aug 10 18:48:07 dist-test-slave-cbsf krb5kdc[19113](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260810 18:48:09.016567 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:48:09 dist-test-slave-cbsf krb5kdc[19113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:48:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:48:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.168.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
I20260810 18:48:09.158195 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:43609
--webserver_interface=127.18.168.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:42929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:43609
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:09.287789 19126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:09.288021 19126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:09.288058 19126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:09.291937 19126 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:48:09.291985 19126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:09.292004 19126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:09.292019 19126 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:48:09.292033 19126 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:48:09.297183 19126 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:42929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:43609
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:43609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19126
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:09.298187 19126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:09.299150 19126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:09.308521 19132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:09.308600 19131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:09.308537 19134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:09.308806 19126 server_base.cc:1034] running on GCE node
I20260810 18:48:09.309293 19126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:09.310652 19126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:09.311820 19126 hybrid_clock.cc:648] HybridClock initialized: now 1786387689311782 us; error 48 us; skew 500 ppm
Aug 10 18:48:09 dist-test-slave-cbsf krb5kdc[19113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387689, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:09.314956 19126 init.cc:377] Logged in from keytab as kudu/127.18.168.190@KRBTEST.COM (short username kudu)
I20260810 18:48:09.315869 19126 webserver.cc:460] Webserver started at http://127.18.168.190:38629/ using document root <none> and password file <none>
I20260810 18:48:09.316489 19126 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:09.316577 19126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:09.316918 19126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:09.319005 19126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "9ab5c3ce04814671986e76041b6a5300"
format_stamp: "Formatted at 2026-08-10 18:48:09 on dist-test-slave-cbsf"
server_key: "6b7c193d99995d9c9b78909c7f287558"
server_key_iv: "393d49554bdf24a8aeb78decadbd7efa"
server_key_version: "encryptionkey@0"
I20260810 18:48:09.319576 19126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "9ab5c3ce04814671986e76041b6a5300"
format_stamp: "Formatted at 2026-08-10 18:48:09 on dist-test-slave-cbsf"
server_key: "6b7c193d99995d9c9b78909c7f287558"
server_key_iv: "393d49554bdf24a8aeb78decadbd7efa"
server_key_version: "encryptionkey@0"
I20260810 18:48:09.323400 19126 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260810 18:48:09.325646 19142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:09.326807 19126 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:48:09.326943 19126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "9ab5c3ce04814671986e76041b6a5300"
format_stamp: "Formatted at 2026-08-10 18:48:09 on dist-test-slave-cbsf"
server_key: "6b7c193d99995d9c9b78909c7f287558"
server_key_iv: "393d49554bdf24a8aeb78decadbd7efa"
server_key_version: "encryptionkey@0"
I20260810 18:48:09.327049 19126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:09.338485 19126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:09.341245 19126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:09.341449 19126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:09.350391 19126 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:43609
I20260810 18:48:09.350405 19197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:43609 every 8 connection(s)
I20260810 18:48:09.351397 19126 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:48:09.354830 19198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:09.354866 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 19126
I20260810 18:48:09.355014 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:48:09.356954 19106 external_mini_cluster.cc:1428] Setting key 41563317b3b377b6b152bab655025f72
I20260810 18:48:09.362083 19198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ab5c3ce04814671986e76041b6a5300: Bootstrap starting.
I20260810 18:48:09.364202 19198 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ab5c3ce04814671986e76041b6a5300: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:09.365017 19198 log.cc:824] T 00000000000000000000000000000000 P 9ab5c3ce04814671986e76041b6a5300: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:09.367136 19198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ab5c3ce04814671986e76041b6a5300: No bootstrap required, opened a new log
I20260810 18:48:09.371289 19198 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ab5c3ce04814671986e76041b6a5300 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ab5c3ce04814671986e76041b6a5300" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43609 } }
I20260810 18:48:09.371558 19198 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ab5c3ce04814671986e76041b6a5300 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:09.371608 19198 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ab5c3ce04814671986e76041b6a5300 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ab5c3ce04814671986e76041b6a5300, State: Initialized, Role: FOLLOWER
I20260810 18:48:09.372104 19198 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ab5c3ce04814671986e76041b6a5300 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ab5c3ce04814671986e76041b6a5300" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43609 } }
I20260810 18:48:09.372234 19198 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ab5c3ce04814671986e76041b6a5300 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:48:09.372323 19198 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ab5c3ce04814671986e76041b6a5300 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:48:09.372445 19198 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ab5c3ce04814671986e76041b6a5300 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:09.373531 19198 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ab5c3ce04814671986e76041b6a5300 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ab5c3ce04814671986e76041b6a5300" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43609 } }
I20260810 18:48:09.373903 19198 leader_election.cc:302] T 00000000000000000000000000000000 P 9ab5c3ce04814671986e76041b6a5300 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ab5c3ce04814671986e76041b6a5300; no voters:
I20260810 18:48:09.374267 19198 leader_election.cc:288] T 00000000000000000000000000000000 P 9ab5c3ce04814671986e76041b6a5300 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:48:09.374356 19203 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ab5c3ce04814671986e76041b6a5300 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:09.374575 19203 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ab5c3ce04814671986e76041b6a5300 [term 1 LEADER]: Becoming Leader. State: Replica: 9ab5c3ce04814671986e76041b6a5300, State: Running, Role: LEADER
I20260810 18:48:09.374927 19203 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ab5c3ce04814671986e76041b6a5300 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ab5c3ce04814671986e76041b6a5300" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43609 } }
I20260810 18:48:09.375636 19198 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ab5c3ce04814671986e76041b6a5300 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:48:09.376938 19204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ab5c3ce04814671986e76041b6a5300 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ab5c3ce04814671986e76041b6a5300. Latest consensus state: current_term: 1 leader_uuid: "9ab5c3ce04814671986e76041b6a5300" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ab5c3ce04814671986e76041b6a5300" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43609 } } }
I20260810 18:48:09.377090 19204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ab5c3ce04814671986e76041b6a5300 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:09.377327 19205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ab5c3ce04814671986e76041b6a5300 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ab5c3ce04814671986e76041b6a5300" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ab5c3ce04814671986e76041b6a5300" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43609 } } }
I20260810 18:48:09.377437 19205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ab5c3ce04814671986e76041b6a5300 [sys.catalog]: This master's current role is: LEADER
Aug 10 18:48:09 dist-test-slave-cbsf krb5kdc[19113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
W20260810 18:48:09.381783 19221 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9ab5c3ce04814671986e76041b6a5300: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:48:09.381860 19221 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:48:09.381932 19213 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:48:09.382664 19213 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:48:09.386196 19200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:09.360738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50106 (local address 127.18.168.190:43609)
0810 18:48:09.361382 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:09.361404 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:09.362401 (+ 997us) server_negotiation.cc:407] Connection header received
0810 18:48:09.366645 (+ 4244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:09.366688 (+ 43us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:09.367048 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:09.367423 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:09.369951 (+ 2528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:09.371612 (+ 1661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:09.373136 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:09.373854 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:09.374027 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:09.379797 (+ 5770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:09.379804 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:09.379820 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:09.379851 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:09.382412 (+ 2561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:09.383050 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:09.383056 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:09.383062 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:09.383193 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:09.383556 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:09.383559 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:09.383561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:09.384026 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:09.384260 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:09.384900 (+ 640us) server_negotiation.cc:299] Negotiation successful
0810 18:48:09.385184 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":252,"threads_started":1}
I20260810 18:48:09.389735 19213 catalog_manager.cc:1347] Generated new cluster ID: 9576cbf93f424b53a93f662b89626572
I20260810 18:48:09.389809 19213 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:48:09.402612 19213 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:48:09.403491 19213 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:48:09.413519 19213 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ab5c3ce04814671986e76041b6a5300: Generated new TSK 0
I20260810 18:48:09.414170 19213 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.168.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab.
Entry for principal kudu/127.18.168.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab.
I20260810 18:48:09.453243 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.129:0
--local_ip_for_outbound_sockets=127.18.168.129
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:43609
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:42929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:09.600032 19229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:09.600325 19229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:09.600411 19229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:09.605108 19229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:09.605185 19229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:09.605278 19229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.129
I20260810 18:48:09.610760 19229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:42929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:43609
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19229
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.129
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:09.611786 19229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:09.612937 19229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:09.623065 19235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:09.623070 19238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:09.623169 19236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:09.623695 19229 server_base.cc:1034] running on GCE node
I20260810 18:48:09.624166 19229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:09.625033 19229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:09.626241 19229 hybrid_clock.cc:648] HybridClock initialized: now 1786387689626217 us; error 40 us; skew 500 ppm
Aug 10 18:48:09 dist-test-slave-cbsf krb5kdc[19113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387689, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:09.628975 19229 init.cc:377] Logged in from keytab as kudu/127.18.168.129@KRBTEST.COM (short username kudu)
I20260810 18:48:09.629917 19229 webserver.cc:460] Webserver started at http://127.18.168.129:41179/ using document root <none> and password file <none>
I20260810 18:48:09.630574 19229 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:09.630664 19229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:09.630880 19229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:09.633019 19229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-0/data/instance:
uuid: "f211e42c65b64adc9acbf3dbcd97ab33"
format_stamp: "Formatted at 2026-08-10 18:48:09 on dist-test-slave-cbsf"
server_key: "c8141cb5705f802f7931cf052eb7f904"
server_key_iv: "e38074a9327b557bdc863133e9e9bf69"
server_key_version: "encryptionkey@0"
I20260810 18:48:09.633682 19229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance:
uuid: "f211e42c65b64adc9acbf3dbcd97ab33"
format_stamp: "Formatted at 2026-08-10 18:48:09 on dist-test-slave-cbsf"
server_key: "c8141cb5705f802f7931cf052eb7f904"
server_key_iv: "e38074a9327b557bdc863133e9e9bf69"
server_key_version: "encryptionkey@0"
I20260810 18:48:09.637635 19229 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:48:09.640080 19247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:09.641311 19229 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:48:09.641451 19229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-0/wal
uuid: "f211e42c65b64adc9acbf3dbcd97ab33"
format_stamp: "Formatted at 2026-08-10 18:48:09 on dist-test-slave-cbsf"
server_key: "c8141cb5705f802f7931cf052eb7f904"
server_key_iv: "e38074a9327b557bdc863133e9e9bf69"
server_key_version: "encryptionkey@0"
I20260810 18:48:09.641561 19229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:09.671984 19229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:09.675053 19229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:09.675352 19229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:09.676095 19229 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:09.677186 19229 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:09.677289 19229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:09.677353 19229 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:09.677402 19229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:09.693140 19229 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.129:45937
I20260810 18:48:09.693207 19373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.129:45937 every 8 connection(s)
I20260810 18:48:09.694082 19229 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
I20260810 18:48:09.702123 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 19229
I20260810 18:48:09.702239 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance
I20260810 18:48:09.702564 19106 external_mini_cluster.cc:1428] Setting key e23e369f5a75aa05531be52f049dd32e
Aug 10 18:48:09 dist-test-slave-cbsf krb5kdc[19113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387689, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:09.709879 19200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:09.695936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:45917 (local address 127.18.168.190:43609)
0810 18:48:09.696086 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:09.696091 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:09.697491 (+ 1400us) server_negotiation.cc:407] Connection header received
0810 18:48:09.698552 (+ 1061us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:09.698554 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:09.698601 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:09.698677 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:09.700080 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:09.700624 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:09.701950 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:09.702195 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:09.702302 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:09.705467 (+ 3165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:09.705473 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:09.705478 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:09.705510 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:09.707315 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:09.708054 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:09.708057 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:09.708058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:09.708136 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:09.708575 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:09.708577 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:09.708579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:09.708815 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:09.708895 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:09.709489 (+ 594us) server_negotiation.cc:299] Negotiation successful
0810 18:48:09.709636 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 18:48:09.710589 19377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:09.696410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43609 (local address 127.18.168.129:45917)
0810 18:48:09.697321 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:09.697361 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:09.698340 (+ 979us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:09.698836 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:09.698842 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:09.699299 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:09.699901 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:09.699917 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:09.700807 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:09.700810 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:09.701806 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:09.701816 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:09.702380 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:09.702385 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:09.702659 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:09.703374 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:09.703393 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:09.705304 (+ 1911us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:09.707442 (+ 2138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:09.707450 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:09.707469 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:09.707910 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:09.708251 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:09.708256 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:09.708259 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:09.708459 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:09.708920 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:09.708925 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:09.709191 (+ 266us) client_negotiation.cc:769] Sending connection context
0810 18:48:09.709366 (+ 175us) client_negotiation.cc:240] Negotiation successful
0810 18:48:09.709644 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"thread_start_us":205,"threads_started":1}
I20260810 18:48:09.711995 19374 heartbeater.cc:347] Connected to a master server at 127.18.168.190:43609
I20260810 18:48:09.712280 19374 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:09.713032 19374 heartbeater.cc:511] Master 127.18.168.190:43609 requested a full tablet report, sending...
I20260810 18:48:09.714949 19160 ts_manager.cc:194] Registered new tserver with Master: f211e42c65b64adc9acbf3dbcd97ab33 (127.18.168.129:45937)
WARNING: no policy specified for kudu/127.18.168.130@KRBTEST.COM; defaulting to no policy
I20260810 18:48:09.716594 19160 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.129@KRBTEST.COM'} at 127.18.168.129:45917
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab.
Entry for principal kudu/127.18.168.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab.
I20260810 18:48:09.739758 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.130:0
--local_ip_for_outbound_sockets=127.18.168.130
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:43609
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:42929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:09.899184 19380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:09.899443 19380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:09.899492 19380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:09.903467 19380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:09.903517 19380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:09.903573 19380 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.130
I20260810 18:48:09.909317 19380 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:42929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:43609
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.130
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:09.910312 19380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:09.911383 19380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:09.922628 19388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:09.922592 19386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:09.922699 19385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:09.922925 19380 server_base.cc:1034] running on GCE node
I20260810 18:48:09.923367 19380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:09.924031 19380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:09.925220 19380 hybrid_clock.cc:648] HybridClock initialized: now 1786387689925208 us; error 28 us; skew 500 ppm
Aug 10 18:48:09 dist-test-slave-cbsf krb5kdc[19113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387689, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:09.928399 19380 init.cc:377] Logged in from keytab as kudu/127.18.168.130@KRBTEST.COM (short username kudu)
I20260810 18:48:09.929301 19380 webserver.cc:460] Webserver started at http://127.18.168.130:33431/ using document root <none> and password file <none>
I20260810 18:48:09.929819 19380 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:09.929877 19380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:09.930039 19380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:09.932119 19380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-1/data/instance:
uuid: "606dae4b1dbc41afaed01b6a1ae43a24"
format_stamp: "Formatted at 2026-08-10 18:48:09 on dist-test-slave-cbsf"
server_key: "ec5b52cd80fc33aebcbf9067742d4f11"
server_key_iv: "189ac3b4d49be24bb1537be427165519"
server_key_version: "encryptionkey@0"
I20260810 18:48:09.932762 19380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance:
uuid: "606dae4b1dbc41afaed01b6a1ae43a24"
format_stamp: "Formatted at 2026-08-10 18:48:09 on dist-test-slave-cbsf"
server_key: "ec5b52cd80fc33aebcbf9067742d4f11"
server_key_iv: "189ac3b4d49be24bb1537be427165519"
server_key_version: "encryptionkey@0"
I20260810 18:48:09.936532 19380 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260810 18:48:09.939011 19396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:09.940253 19380 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:48:09.940373 19380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-1/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-1/wal
uuid: "606dae4b1dbc41afaed01b6a1ae43a24"
format_stamp: "Formatted at 2026-08-10 18:48:09 on dist-test-slave-cbsf"
server_key: "ec5b52cd80fc33aebcbf9067742d4f11"
server_key_iv: "189ac3b4d49be24bb1537be427165519"
server_key_version: "encryptionkey@0"
I20260810 18:48:09.940472 19380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:09.950209 19380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:09.953531 19380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:09.953850 19380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:09.954666 19380 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:09.955549 19380 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:09.955612 19380 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:09.955681 19380 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:09.955713 19380 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:09.974504 19380 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.130:41613
I20260810 18:48:09.974530 19519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.130:41613 every 8 connection(s)
I20260810 18:48:09.975453 19380 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
I20260810 18:48:09.976936 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 19380
I20260810 18:48:09.977048 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance
I20260810 18:48:09.977342 19106 external_mini_cluster.cc:1428] Setting key c67178e7aad619849695ba4d5e07653b
Aug 10 18:48:09 dist-test-slave-cbsf krb5kdc[19113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387689, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.168.131@KRBTEST.COM; defaulting to no policy
I20260810 18:48:09.993253 19200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:09.977719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:35671 (local address 127.18.168.190:43609)
0810 18:48:09.977879 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:09.977884 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:09.979632 (+ 1748us) server_negotiation.cc:407] Connection header received
0810 18:48:09.981416 (+ 1784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:09.981419 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:09.981463 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:09.981534 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:09.983365 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:09.983954 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:09.985481 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:09.985928 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:09.986069 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:09.989238 (+ 3169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:09.989242 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:09.989245 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:09.989274 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:09.991028 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:09.991706 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:09.991710 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:09.991712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:09.991801 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:09.992129 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:09.992131 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:09.992132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:09.992325 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:09.992399 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:09.992952 (+ 553us) server_negotiation.cc:299] Negotiation successful
0810 18:48:09.993052 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.131@KRBTEST.COM" created.
I20260810 18:48:09.994204 19522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:09.978188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43609 (local address 127.18.168.130:35671)
0810 18:48:09.979419 (+ 1231us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:09.979510 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:09.981163 (+ 1653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:09.981821 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:09.981841 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:09.982550 (+ 709us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:09.983190 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:09.983206 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:09.984085 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:09.984088 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:09.985340 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:09.985351 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:09.986102 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:09.986105 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:09.986322 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:09.987062 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:09.987080 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:09.989050 (+ 1970us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:09.991166 (+ 2116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:09.991171 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:09.991183 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:09.991569 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:09.991893 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:09.991895 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:09.991896 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:09.992040 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:09.992417 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:09.992422 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:09.992657 (+ 235us) client_negotiation.cc:769] Sending connection context
0810 18:48:09.992872 (+ 215us) client_negotiation.cc:240] Negotiation successful
0810 18:48:09.993133 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":831,"thread_start_us":241,"threads_started":1}
I20260810 18:48:09.995555 19520 heartbeater.cc:347] Connected to a master server at 127.18.168.190:43609
I20260810 18:48:09.995841 19520 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:09.996387 19520 heartbeater.cc:511] Master 127.18.168.190:43609 requested a full tablet report, sending...
I20260810 18:48:09.997838 19160 ts_manager.cc:194] Registered new tserver with Master: 606dae4b1dbc41afaed01b6a1ae43a24 (127.18.168.130:41613)
I20260810 18:48:09.998497 19160 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.130@KRBTEST.COM'} at 127.18.168.130:35671
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab.
Entry for principal kudu/127.18.168.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab.
I20260810 18:48:10.015587 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.131:0
--local_ip_for_outbound_sockets=127.18.168.131
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:43609
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:42929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:10.167294 19525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:10.167562 19525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:10.167605 19525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:10.171859 19525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:10.171923 19525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:10.172051 19525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.131
I20260810 18:48:10.177825 19525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:42929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:43609
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.131
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:10.178848 19525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:10.179883 19525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:10.190586 19533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:10.190573 19531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:10.190624 19530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:10.190815 19525 server_base.cc:1034] running on GCE node
I20260810 18:48:10.191221 19525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:10.191834 19525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:10.193037 19525 hybrid_clock.cc:648] HybridClock initialized: now 1786387690193007 us; error 41 us; skew 500 ppm
Aug 10 18:48:10 dist-test-slave-cbsf krb5kdc[19113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387690, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:10.195734 19525 init.cc:377] Logged in from keytab as kudu/127.18.168.131@KRBTEST.COM (short username kudu)
I20260810 18:48:10.196534 19525 webserver.cc:460] Webserver started at http://127.18.168.131:43333/ using document root <none> and password file <none>
I20260810 18:48:10.197168 19525 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:10.197269 19525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:10.197549 19525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:10.199545 19525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-2/data/instance:
uuid: "8c379d6f38164612b6a98783cf3b3a7d"
format_stamp: "Formatted at 2026-08-10 18:48:10 on dist-test-slave-cbsf"
server_key: "2f1da41f1ecd329fa3d79dc0ed7ca661"
server_key_iv: "52353807455d30c0288c2720da16f525"
server_key_version: "encryptionkey@0"
I20260810 18:48:10.200105 19525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance:
uuid: "8c379d6f38164612b6a98783cf3b3a7d"
format_stamp: "Formatted at 2026-08-10 18:48:10 on dist-test-slave-cbsf"
server_key: "2f1da41f1ecd329fa3d79dc0ed7ca661"
server_key_iv: "52353807455d30c0288c2720da16f525"
server_key_version: "encryptionkey@0"
I20260810 18:48:10.204031 19525 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260810 18:48:10.206452 19540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:10.207701 19525 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260810 18:48:10.207851 19525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-2/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-2/wal
uuid: "8c379d6f38164612b6a98783cf3b3a7d"
format_stamp: "Formatted at 2026-08-10 18:48:10 on dist-test-slave-cbsf"
server_key: "2f1da41f1ecd329fa3d79dc0ed7ca661"
server_key_iv: "52353807455d30c0288c2720da16f525"
server_key_version: "encryptionkey@0"
I20260810 18:48:10.207962 19525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:10.222340 19525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:10.225274 19525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:10.225479 19525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:10.226099 19525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:10.227082 19525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:10.227160 19525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:10.227252 19525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:10.227303 19525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:10.247416 19525 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.131:46147
I20260810 18:48:10.247495 19666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.131:46147 every 8 connection(s)
I20260810 18:48:10.248490 19525 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
I20260810 18:48:10.252979 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 19525
I20260810 18:48:10.253086 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance
I20260810 18:48:10.253384 19106 external_mini_cluster.cc:1428] Setting key 05378e3534e718b589fdb7eac7568c4b
Aug 10 18:48:10 dist-test-slave-cbsf krb5kdc[19113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387690, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:10.264465 19200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:10.250679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:42177 (local address 127.18.168.190:43609)
0810 18:48:10.250830 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:10.250833 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:10.252129 (+ 1296us) server_negotiation.cc:407] Connection header received
0810 18:48:10.253443 (+ 1314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:10.253446 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:10.253509 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:10.253586 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:10.255482 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:10.256086 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:10.257686 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:10.257902 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:10.257984 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:10.260851 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:10.260854 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:10.260856 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:10.260879 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:10.262407 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:10.262934 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:10.262943 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:10.262944 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:10.263018 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:10.263391 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:10.263394 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:10.263396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:10.263573 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:10.263706 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:10.264197 (+ 491us) server_negotiation.cc:299] Negotiation successful
0810 18:48:10.264311 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 18:48:10.265483 19670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:10.251068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43609 (local address 127.18.168.131:42177)
0810 18:48:10.251941 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:10.251984 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:10.253227 (+ 1243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:10.253772 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:10.253779 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:10.254383 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:10.255332 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:10.255351 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:10.256270 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:10.256273 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:10.257550 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:10.257560 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:10.258038 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:10.258041 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:10.258245 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:10.258888 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:10.258913 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:10.260618 (+ 1705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:10.262540 (+ 1922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:10.262546 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:10.262560 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:10.262820 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:10.263112 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:10.263116 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:10.263118 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:10.263291 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:10.263717 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:10.263726 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:10.263958 (+ 232us) client_negotiation.cc:769] Sending connection context
0810 18:48:10.264138 (+ 180us) client_negotiation.cc:240] Negotiation successful
0810 18:48:10.264488 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":192,"threads_started":1}
I20260810 18:48:10.266793 19668 heartbeater.cc:347] Connected to a master server at 127.18.168.190:43609
I20260810 18:48:10.267082 19668 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:10.267644 19668 heartbeater.cc:511] Master 127.18.168.190:43609 requested a full tablet report, sending...
I20260810 18:48:10.268875 19160 ts_manager.cc:194] Registered new tserver with Master: 8c379d6f38164612b6a98783cf3b3a7d (127.18.168.131:46147)
I20260810 18:48:10.269497 19160 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.131@KRBTEST.COM'} at 127.18.168.131:42177
I20260810 18:48:10.280244 19106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 18:48:10 dist-test-slave-cbsf krb5kdc[19113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387690, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 18:48:10 dist-test-slave-cbsf krb5kdc[19113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387690, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.168.129@KRBTEST.COM
I20260810 18:48:10.309965 19675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:10.299920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57570 (local address 127.18.168.129:45937)
0810 18:48:10.300259 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:10.300267 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:10.300297 (+ 30us) server_negotiation.cc:407] Connection header received
0810 18:48:10.300354 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:10.300360 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:10.300563 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:10.300803 (+ 240us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:10.301597 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:10.302640 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:10.303486 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:10.303919 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:10.304012 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:10.306548 (+ 2536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:10.306558 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:10.306571 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:10.306596 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:10.308262 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:10.308731 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:10.308741 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:10.308742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:10.308823 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:10.309075 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:10.309077 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:10.309078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:10.309304 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:10.309454 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:10.309673 (+ 219us) server_negotiation.cc:299] Negotiation successful
0810 18:48:10.309795 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":109,"threads_started":1}
W20260810 18:48:10.311053 19279 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57570
Aug 10 18:48:10 dist-test-slave-cbsf krb5kdc[19113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 18:48:10 dist-test-slave-cbsf krb5kdc[19113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.168.129@KRBTEST.COM
I20260810 18:48:10.338605 19675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:10.330740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60512 (local address 127.18.168.129:45937)
0810 18:48:10.330904 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:10.330907 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:10.331086 (+ 179us) server_negotiation.cc:407] Connection header received
0810 18:48:10.331193 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:10.331195 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:10.331235 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:10.331308 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:10.332053 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:10.332559 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:10.333410 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:10.333632 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:10.333730 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:10.336005 (+ 2275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:10.336007 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:10.336009 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:10.336030 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:10.337277 (+ 1247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:10.337657 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:10.337660 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:10.337661 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:10.337731 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:10.337955 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:10.337956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:10.337957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:10.338105 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:10.338206 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:10.338388 (+ 182us) server_negotiation.cc:299] Negotiation successful
0810 18:48:10.338478 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260810 18:48:10.339627 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 19229
I20260810 18:48:10.350730 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 19380
I20260810 18:48:10.359133 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 19525
I20260810 18:48:10.367678 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 19126
2026-08-10T18:48:10Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2505 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:48:10 dist-test-slave-cbsf krb5kdc[19682](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:48:10 dist-test-slave-cbsf krb5kdc[19682](info): set up 2 sockets
Aug 10 18:48:10 dist-test-slave-cbsf krb5kdc[19682](info): commencing operation
krb5kdc: starting...
W20260810 18:48:11.478895 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:48:11 dist-test-slave-cbsf krb5kdc[19682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:48:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:48:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.168.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
I20260810 18:48:11.601459 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:42239
--webserver_interface=127.18.168.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:36993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:42239
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:11.718041 19697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:11.718305 19697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:11.718349 19697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:11.722510 19697 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:48:11.722558 19697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:11.722576 19697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:11.722592 19697 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:48:11.722607 19697 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:48:11.727646 19697 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:36993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:42239
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:42239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:11.728555 19697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:11.729498 19697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:11.737732 19706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:11.737711 19703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:11.737712 19704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:11.738073 19697 server_base.cc:1034] running on GCE node
I20260810 18:48:11.738584 19697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:11.739616 19697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:11.740780 19697 hybrid_clock.cc:648] HybridClock initialized: now 1786387691740750 us; error 40 us; skew 500 ppm
Aug 10 18:48:11 dist-test-slave-cbsf krb5kdc[19682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387691, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:11.743498 19697 init.cc:377] Logged in from keytab as kudu/127.18.168.190@KRBTEST.COM (short username kudu)
I20260810 18:48:11.744379 19697 webserver.cc:460] Webserver started at http://127.18.168.190:37613/ using document root <none> and password file <none>
I20260810 18:48:11.744998 19697 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:11.745083 19697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:11.745306 19697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:11.747293 19697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "6dcc2c4b974541d6a62513d8b2e16ed1"
format_stamp: "Formatted at 2026-08-10 18:48:11 on dist-test-slave-cbsf"
server_key: "114d67067f60be26c4b5459b83f2ae78"
server_key_iv: "255397c3dd44c91374ca827111daac6d"
server_key_version: "encryptionkey@0"
I20260810 18:48:11.747862 19697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "6dcc2c4b974541d6a62513d8b2e16ed1"
format_stamp: "Formatted at 2026-08-10 18:48:11 on dist-test-slave-cbsf"
server_key: "114d67067f60be26c4b5459b83f2ae78"
server_key_iv: "255397c3dd44c91374ca827111daac6d"
server_key_version: "encryptionkey@0"
I20260810 18:48:11.751499 19697 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260810 18:48:11.753791 19713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:11.754968 19697 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:48:11.755110 19697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "6dcc2c4b974541d6a62513d8b2e16ed1"
format_stamp: "Formatted at 2026-08-10 18:48:11 on dist-test-slave-cbsf"
server_key: "114d67067f60be26c4b5459b83f2ae78"
server_key_iv: "255397c3dd44c91374ca827111daac6d"
server_key_version: "encryptionkey@0"
I20260810 18:48:11.755214 19697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:11.775090 19697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:11.778098 19697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:11.778303 19697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:11.785400 19697 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:42239
I20260810 18:48:11.785414 19774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:42239 every 8 connection(s)
I20260810 18:48:11.786338 19697 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:48:11.787514 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 19697
I20260810 18:48:11.787628 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:48:11.787950 19106 external_mini_cluster.cc:1428] Setting key 3b674d2c554a940cee9f6fb1a9d88452
I20260810 18:48:11.789815 19775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:11.796669 19775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6dcc2c4b974541d6a62513d8b2e16ed1: Bootstrap starting.
Aug 10 18:48:11 dist-test-slave-cbsf krb5kdc[19682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:11.799499 19775 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6dcc2c4b974541d6a62513d8b2e16ed1: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:11.800449 19775 log.cc:824] T 00000000000000000000000000000000 P 6dcc2c4b974541d6a62513d8b2e16ed1: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:11.802443 19775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6dcc2c4b974541d6a62513d8b2e16ed1: No bootstrap required, opened a new log
I20260810 18:48:11.803068 19778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:11.789354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60552 (local address 127.18.168.190:42239)
0810 18:48:11.789779 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:11.789799 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:11.789843 (+ 44us) server_negotiation.cc:407] Connection header received
0810 18:48:11.790675 (+ 832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:11.790697 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:11.791080 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:11.791444 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:11.792124 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:11.793678 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:11.794494 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:11.795215 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:11.795383 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:11.797906 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:11.797912 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:11.797930 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:11.797967 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:11.799925 (+ 1958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:11.800343 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:11.800349 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:11.800354 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:11.800526 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:11.800910 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:11.800914 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:11.800916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:11.801318 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:11.801522 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:11.801848 (+ 326us) server_negotiation.cc:299] Negotiation successful
0810 18:48:11.802118 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":126,"threads_started":1}
I20260810 18:48:11.805909 19775 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6dcc2c4b974541d6a62513d8b2e16ed1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dcc2c4b974541d6a62513d8b2e16ed1" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 42239 } }
I20260810 18:48:11.806111 19775 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6dcc2c4b974541d6a62513d8b2e16ed1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:11.806201 19775 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6dcc2c4b974541d6a62513d8b2e16ed1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dcc2c4b974541d6a62513d8b2e16ed1, State: Initialized, Role: FOLLOWER
I20260810 18:48:11.806630 19775 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6dcc2c4b974541d6a62513d8b2e16ed1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dcc2c4b974541d6a62513d8b2e16ed1" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 42239 } }
I20260810 18:48:11.806777 19775 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6dcc2c4b974541d6a62513d8b2e16ed1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:48:11.806844 19775 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6dcc2c4b974541d6a62513d8b2e16ed1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:48:11.806941 19775 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6dcc2c4b974541d6a62513d8b2e16ed1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:11.808007 19775 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6dcc2c4b974541d6a62513d8b2e16ed1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dcc2c4b974541d6a62513d8b2e16ed1" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 42239 } }
I20260810 18:48:11.808385 19775 leader_election.cc:302] T 00000000000000000000000000000000 P 6dcc2c4b974541d6a62513d8b2e16ed1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6dcc2c4b974541d6a62513d8b2e16ed1; no voters:
I20260810 18:48:11.808686 19775 leader_election.cc:288] T 00000000000000000000000000000000 P 6dcc2c4b974541d6a62513d8b2e16ed1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:48:11.808856 19780 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6dcc2c4b974541d6a62513d8b2e16ed1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:11.809098 19780 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6dcc2c4b974541d6a62513d8b2e16ed1 [term 1 LEADER]: Becoming Leader. State: Replica: 6dcc2c4b974541d6a62513d8b2e16ed1, State: Running, Role: LEADER
I20260810 18:48:11.809406 19780 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6dcc2c4b974541d6a62513d8b2e16ed1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dcc2c4b974541d6a62513d8b2e16ed1" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 42239 } }
I20260810 18:48:11.810022 19775 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6dcc2c4b974541d6a62513d8b2e16ed1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:48:11.811089 19782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6dcc2c4b974541d6a62513d8b2e16ed1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6dcc2c4b974541d6a62513d8b2e16ed1. Latest consensus state: current_term: 1 leader_uuid: "6dcc2c4b974541d6a62513d8b2e16ed1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dcc2c4b974541d6a62513d8b2e16ed1" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 42239 } } }
I20260810 18:48:11.811233 19782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6dcc2c4b974541d6a62513d8b2e16ed1 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:11.811089 19781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6dcc2c4b974541d6a62513d8b2e16ed1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6dcc2c4b974541d6a62513d8b2e16ed1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dcc2c4b974541d6a62513d8b2e16ed1" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 42239 } } }
I20260810 18:48:11.811554 19781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6dcc2c4b974541d6a62513d8b2e16ed1 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:11.811702 19791 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:48:11.814486 19791 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:48:11.820217 19791 catalog_manager.cc:1347] Generated new cluster ID: 525468992e5641f0b29bbd98e39fcd72
I20260810 18:48:11.820295 19791 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:48:11.832474 19791 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:48:11.833635 19791 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:48:11.838726 19791 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6dcc2c4b974541d6a62513d8b2e16ed1: Generated new TSK 0
I20260810 18:48:11.839352 19791 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.168.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab.
Entry for principal kudu/127.18.168.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab.
I20260810 18:48:11.870276 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.129:0
--local_ip_for_outbound_sockets=127.18.168.129
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:42239
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:36993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:11.983368 19807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:11.983681 19807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:11.983769 19807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:11.988070 19807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:11.988178 19807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:11.988270 19807 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.129
I20260810 18:48:11.993804 19807 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:36993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:42239
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19807
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.129
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:11.994858 19807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:11.995855 19807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:12.005321 19814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:12.005308 19815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:12.005463 19819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:12.005527 19807 server_base.cc:1034] running on GCE node
I20260810 18:48:12.005949 19807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:12.006559 19807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:12.007766 19807 hybrid_clock.cc:648] HybridClock initialized: now 1786387692007733 us; error 30 us; skew 500 ppm
Aug 10 18:48:12 dist-test-slave-cbsf krb5kdc[19682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387692, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:12.011505 19807 init.cc:377] Logged in from keytab as kudu/127.18.168.129@KRBTEST.COM (short username kudu)
I20260810 18:48:12.012538 19807 webserver.cc:460] Webserver started at http://127.18.168.129:44973/ using document root <none> and password file <none>
I20260810 18:48:12.013159 19807 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:12.013250 19807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:12.013463 19807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:12.015432 19807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-0/data/instance:
uuid: "7c74e5c9f43e4e3d8518a83d43004c0a"
format_stamp: "Formatted at 2026-08-10 18:48:12 on dist-test-slave-cbsf"
server_key: "3f2c4d19e6b71fda3efefb61e118ef64"
server_key_iv: "63858fce71dc1c048666fc82b5cb5abc"
server_key_version: "encryptionkey@0"
I20260810 18:48:12.016016 19807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance:
uuid: "7c74e5c9f43e4e3d8518a83d43004c0a"
format_stamp: "Formatted at 2026-08-10 18:48:12 on dist-test-slave-cbsf"
server_key: "3f2c4d19e6b71fda3efefb61e118ef64"
server_key_iv: "63858fce71dc1c048666fc82b5cb5abc"
server_key_version: "encryptionkey@0"
I20260810 18:48:12.019697 19807 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260810 18:48:12.022101 19827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:12.023283 19807 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:48:12.023417 19807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-0/wal
uuid: "7c74e5c9f43e4e3d8518a83d43004c0a"
format_stamp: "Formatted at 2026-08-10 18:48:12 on dist-test-slave-cbsf"
server_key: "3f2c4d19e6b71fda3efefb61e118ef64"
server_key_iv: "63858fce71dc1c048666fc82b5cb5abc"
server_key_version: "encryptionkey@0"
I20260810 18:48:12.023530 19807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:12.055898 19807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:12.059441 19807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:12.059669 19807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:12.060298 19807 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:12.061146 19807 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:12.061218 19807 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:12.061285 19807 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:12.061335 19807 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:12.072963 19807 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.129:39263
I20260810 18:48:12.073129 19967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.129:39263 every 8 connection(s)
I20260810 18:48:12.073964 19807 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
I20260810 18:48:12.077317 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 19807
I20260810 18:48:12.077433 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance
I20260810 18:48:12.077831 19106 external_mini_cluster.cc:1428] Setting key 15066733cc9d35f014d4d14bcb32c54e
Aug 10 18:48:12 dist-test-slave-cbsf krb5kdc[19682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387692, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.168.130@KRBTEST.COM; defaulting to no policy
I20260810 18:48:12.090353 19778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:12.075895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:47107 (local address 127.18.168.190:42239)
0810 18:48:12.076114 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:12.076119 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:12.077107 (+ 988us) server_negotiation.cc:407] Connection header received
0810 18:48:12.078919 (+ 1812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:12.078922 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:12.078967 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:12.079070 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:12.080804 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:12.081322 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:12.082736 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:12.083170 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:12.083296 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:12.086325 (+ 3029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:12.086331 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:12.086336 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:12.086373 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:12.088194 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:12.088764 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:12.088771 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:12.088775 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:12.088872 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:12.089209 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:12.089215 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:12.089219 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:12.089501 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:12.089611 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:12.090086 (+ 475us) server_negotiation.cc:299] Negotiation successful
0810 18:48:12.090233 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260810 18:48:12.091306 19970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:12.076303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:42239 (local address 127.18.168.129:47107)
0810 18:48:12.076926 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:12.076968 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:12.078701 (+ 1733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:12.079259 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:12.079268 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:12.079777 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:12.080581 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:12.080604 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:12.081503 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:12.081509 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:12.082603 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:12.082614 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:12.083344 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:12.083347 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:12.083559 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:12.084220 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:12.084236 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:12.086143 (+ 1907us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:12.088327 (+ 2184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:12.088331 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:12.088343 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:12.088588 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:12.088964 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:12.088966 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:12.088967 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:12.089095 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:12.089620 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:12.089625 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:12.089856 (+ 231us) client_negotiation.cc:769] Sending connection context
0810 18:48:12.090002 (+ 146us) client_negotiation.cc:240] Negotiation successful
0810 18:48:12.090330 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":225,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.130@KRBTEST.COM" created.
I20260810 18:48:12.092592 19968 heartbeater.cc:347] Connected to a master server at 127.18.168.190:42239
I20260810 18:48:12.092877 19968 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:12.093405 19968 heartbeater.cc:511] Master 127.18.168.190:42239 requested a full tablet report, sending...
I20260810 18:48:12.094995 19732 ts_manager.cc:194] Registered new tserver with Master: 7c74e5c9f43e4e3d8518a83d43004c0a (127.18.168.129:39263)
I20260810 18:48:12.096158 19732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.129@KRBTEST.COM'} at 127.18.168.129:47107
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab.
Entry for principal kudu/127.18.168.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab.
I20260810 18:48:12.109365 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.130:0
--local_ip_for_outbound_sockets=127.18.168.130
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:42239
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:36993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:12.225674 19973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:12.225931 19973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:12.226006 19973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:12.230058 19973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:12.230139 19973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:12.230230 19973 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.130
I20260810 18:48:12.235159 19973 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:36993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:42239
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19973
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.130
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:12.236135 19973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:12.237144 19973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:12.246877 19983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:12.246934 19980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:12.247033 19973 server_base.cc:1034] running on GCE node
W20260810 18:48:12.246939 19981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:12.247593 19973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:12.248207 19973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:12.249394 19973 hybrid_clock.cc:648] HybridClock initialized: now 1786387692249376 us; error 25 us; skew 500 ppm
Aug 10 18:48:12 dist-test-slave-cbsf krb5kdc[19682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387692, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:12.252112 19973 init.cc:377] Logged in from keytab as kudu/127.18.168.130@KRBTEST.COM (short username kudu)
I20260810 18:48:12.253119 19973 webserver.cc:460] Webserver started at http://127.18.168.130:40539/ using document root <none> and password file <none>
I20260810 18:48:12.253707 19973 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:12.253801 19973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:12.254034 19973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:12.256181 19973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-1/data/instance:
uuid: "82530ed3297e4277b4396d158b7c07d6"
format_stamp: "Formatted at 2026-08-10 18:48:12 on dist-test-slave-cbsf"
server_key: "38397d6cdf7795c0f12fe3dec417c53c"
server_key_iv: "fa5bdda07fd1b59fcb9f2e8f350d6741"
server_key_version: "encryptionkey@0"
I20260810 18:48:12.256808 19973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance:
uuid: "82530ed3297e4277b4396d158b7c07d6"
format_stamp: "Formatted at 2026-08-10 18:48:12 on dist-test-slave-cbsf"
server_key: "38397d6cdf7795c0f12fe3dec417c53c"
server_key_iv: "fa5bdda07fd1b59fcb9f2e8f350d6741"
server_key_version: "encryptionkey@0"
I20260810 18:48:12.260444 19973 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:48:12.262876 19991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:12.264042 19973 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:48:12.264185 19973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-1/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-1/wal
uuid: "82530ed3297e4277b4396d158b7c07d6"
format_stamp: "Formatted at 2026-08-10 18:48:12 on dist-test-slave-cbsf"
server_key: "38397d6cdf7795c0f12fe3dec417c53c"
server_key_iv: "fa5bdda07fd1b59fcb9f2e8f350d6741"
server_key_version: "encryptionkey@0"
I20260810 18:48:12.264288 19973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:12.280239 19973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:12.283161 19973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:12.283311 19973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:12.283828 19973 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:12.284601 19973 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:12.284648 19973 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:12.284685 19973 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:12.284788 19973 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:12.295514 19973 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.130:36143
I20260810 18:48:12.295650 20114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.130:36143 every 8 connection(s)
I20260810 18:48:12.296483 19973 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
I20260810 18:48:12.305888 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 19973
I20260810 18:48:12.306020 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance
I20260810 18:48:12.306356 19106 external_mini_cluster.cc:1428] Setting key 12135746f55dbfeadb05c9f4ee3def16
Aug 10 18:48:12 dist-test-slave-cbsf krb5kdc[19682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387692, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:12.311616 19778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:12.298292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:51319 (local address 127.18.168.190:42239)
0810 18:48:12.298446 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:12.298450 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:12.299465 (+ 1015us) server_negotiation.cc:407] Connection header received
0810 18:48:12.300992 (+ 1527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:12.300995 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:12.301045 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:12.301117 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:12.302495 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:12.303011 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:12.304155 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:12.304358 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:12.304435 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:12.307511 (+ 3076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:12.307514 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:12.307516 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:12.307537 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:12.309239 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:12.309756 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:12.309763 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:12.309767 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:12.309867 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:12.310201 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:12.310206 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:12.310208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:12.310448 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:12.310587 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:12.311270 (+ 683us) server_negotiation.cc:299] Negotiation successful
0810 18:48:12.311416 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 18:48:12.312556 20117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:12.298651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:42239 (local address 127.18.168.130:51319)
0810 18:48:12.299308 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:12.299348 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:12.300807 (+ 1459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:12.301274 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:12.301299 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:12.301769 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:12.302307 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:12.302322 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:12.303135 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:12.303138 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:12.304024 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:12.304033 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:12.304486 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:12.304489 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:12.304668 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:12.305352 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:12.305367 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:12.307334 (+ 1967us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:12.309374 (+ 2040us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:12.309379 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:12.309390 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:12.309668 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:12.309955 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:12.309957 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:12.309958 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:12.310090 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:12.310610 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:12.310618 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:12.311000 (+ 382us) client_negotiation.cc:769] Sending connection context
0810 18:48:12.311219 (+ 219us) client_negotiation.cc:240] Negotiation successful
0810 18:48:12.311559 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":157,"threads_started":1}
I20260810 18:48:12.313905 20115 heartbeater.cc:347] Connected to a master server at 127.18.168.190:42239
I20260810 18:48:12.314177 20115 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:12.314658 20115 heartbeater.cc:511] Master 127.18.168.190:42239 requested a full tablet report, sending...
I20260810 18:48:12.315794 19732 ts_manager.cc:194] Registered new tserver with Master: 82530ed3297e4277b4396d158b7c07d6 (127.18.168.130:36143)
I20260810 18:48:12.316352 19732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.130@KRBTEST.COM'} at 127.18.168.130:51319
WARNING: no policy specified for kudu/127.18.168.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab.
Entry for principal kudu/127.18.168.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab.
I20260810 18:48:12.335464 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.131:0
--local_ip_for_outbound_sockets=127.18.168.131
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:42239
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:36993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:12.458357 20120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:12.458642 20120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:12.458689 20120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:12.463056 20120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:12.463120 20120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:12.463179 20120 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.131
I20260810 18:48:12.468478 20120 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:36993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:42239
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.131
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:12.469569 20120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:12.470548 20120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:12.479923 20127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:12.480023 20126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:12.479979 20129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:12.480233 20120 server_base.cc:1034] running on GCE node
I20260810 18:48:12.480680 20120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:12.481448 20120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:12.482612 20120 hybrid_clock.cc:648] HybridClock initialized: now 1786387692482586 us; error 43 us; skew 500 ppm
Aug 10 18:48:12 dist-test-slave-cbsf krb5kdc[19682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387692, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:12.485808 20120 init.cc:377] Logged in from keytab as kudu/127.18.168.131@KRBTEST.COM (short username kudu)
I20260810 18:48:12.486685 20120 webserver.cc:460] Webserver started at http://127.18.168.131:46111/ using document root <none> and password file <none>
I20260810 18:48:12.487284 20120 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:12.487377 20120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:12.487627 20120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:12.489926 20120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-2/data/instance:
uuid: "2bd84beea3544d6cadda08601474d294"
format_stamp: "Formatted at 2026-08-10 18:48:12 on dist-test-slave-cbsf"
server_key: "ec363f6f08475798553bfbbdd671115b"
server_key_iv: "4adb5535173399f398dc7091ef474c28"
server_key_version: "encryptionkey@0"
I20260810 18:48:12.490510 20120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance:
uuid: "2bd84beea3544d6cadda08601474d294"
format_stamp: "Formatted at 2026-08-10 18:48:12 on dist-test-slave-cbsf"
server_key: "ec363f6f08475798553bfbbdd671115b"
server_key_iv: "4adb5535173399f398dc7091ef474c28"
server_key_version: "encryptionkey@0"
I20260810 18:48:12.494444 20120 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260810 18:48:12.496820 20137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:12.497983 20120 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 18:48:12.498124 20120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-2/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-2/wal
uuid: "2bd84beea3544d6cadda08601474d294"
format_stamp: "Formatted at 2026-08-10 18:48:12 on dist-test-slave-cbsf"
server_key: "ec363f6f08475798553bfbbdd671115b"
server_key_iv: "4adb5535173399f398dc7091ef474c28"
server_key_version: "encryptionkey@0"
I20260810 18:48:12.498245 20120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:12.548969 20120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:12.551851 20120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:12.552060 20120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:12.552678 20120 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:12.553592 20120 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:12.553670 20120 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:12.553771 20120 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:12.553819 20120 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:12.564069 20120 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.131:45545
I20260810 18:48:12.564178 20263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.131:45545 every 8 connection(s)
I20260810 18:48:12.565060 20120 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
I20260810 18:48:12.572139 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 20120
I20260810 18:48:12.572273 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance
I20260810 18:48:12.572615 19106 external_mini_cluster.cc:1428] Setting key c61c1545226d7db27f11d197fc5b3b71
Aug 10 18:48:12 dist-test-slave-cbsf krb5kdc[19682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387692, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:12.579813 19778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:12.566927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:48043 (local address 127.18.168.190:42239)
0810 18:48:12.567081 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:12.567084 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:12.568008 (+ 924us) server_negotiation.cc:407] Connection header received
0810 18:48:12.569396 (+ 1388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:12.569399 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:12.569445 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:12.569519 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:12.570985 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:12.571737 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:12.573364 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:12.573666 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:12.573743 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:12.576543 (+ 2800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:12.576546 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:12.576548 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:12.576569 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:12.578041 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:12.578518 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:12.578521 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:12.578522 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:12.578595 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:12.578887 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:12.578889 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:12.578890 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:12.579050 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:12.579123 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:12.579605 (+ 482us) server_negotiation.cc:299] Negotiation successful
0810 18:48:12.579698 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 18:48:12.580631 20267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:12.567348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:42239 (local address 127.18.168.131:48043)
0810 18:48:12.567857 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:12.567899 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:12.569169 (+ 1270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:12.569690 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:12.569697 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:12.570052 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:12.570809 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:12.570831 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:12.571842 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:12.571844 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:12.573236 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:12.573245 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:12.573805 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:12.573809 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:12.573986 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:12.574552 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:12.574573 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:12.576392 (+ 1819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:12.578151 (+ 1759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:12.578156 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:12.578167 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:12.578418 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:12.578674 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:12.578677 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:12.578678 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:12.578806 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:12.579138 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:12.579143 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:12.579364 (+ 221us) client_negotiation.cc:769] Sending connection context
0810 18:48:12.579559 (+ 195us) client_negotiation.cc:240] Negotiation successful
0810 18:48:12.579795 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":164,"threads_started":1}
I20260810 18:48:12.581745 20264 heartbeater.cc:347] Connected to a master server at 127.18.168.190:42239
I20260810 18:48:12.581951 20264 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:12.582350 20264 heartbeater.cc:511] Master 127.18.168.190:42239 requested a full tablet report, sending...
I20260810 18:48:12.583285 19732 ts_manager.cc:194] Registered new tserver with Master: 2bd84beea3544d6cadda08601474d294 (127.18.168.131:45545)
I20260810 18:48:12.583781 19732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.131@KRBTEST.COM'} at 127.18.168.131:48043
I20260810 18:48:12.586522 19106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:48:12.596486 19778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:12.589984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60568 (local address 127.18.168.190:42239)
0810 18:48:12.590118 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:12.590123 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:12.590285 (+ 162us) server_negotiation.cc:407] Connection header received
0810 18:48:12.590375 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:12.590377 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:12.590426 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:12.590487 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:12.591120 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:12.591587 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:12.592317 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:12.592512 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:12.592586 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:12.593865 (+ 1279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:12.593868 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:12.593870 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:12.593889 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:12.595067 (+ 1178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:12.595425 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:12.595427 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:12.595428 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:12.595497 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:12.595731 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:12.595733 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:12.595734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:12.595910 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:12.596003 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:12.596180 (+ 177us) server_negotiation.cc:299] Negotiation successful
0810 18:48:12.596281 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 18:48:12.601070 19732 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60568:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:48:12.604219 19732 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:48:12.618803 20278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:12.612547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:39263 (local address 127.0.0.1:38052)
0810 18:48:12.613135 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:12.613187 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:12.613316 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:12.614552 (+ 1236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:12.614555 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:12.614571 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:12.614672 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:12.614677 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:12.615899 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:12.615902 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:12.617311 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:12.617322 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:12.618341 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:12.618345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:12.618476 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:12.618510 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 18:48:12.618599 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 18:48:12.618696 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"mutex_wait_us":35,"thread_start_us":96,"threads_started":1}
I20260810 18:48:12.619620 20279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:12.612389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:36143 (local address 127.0.0.1:48998)
0810 18:48:12.613135 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:12.613185 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:12.613438 (+ 253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:12.614169 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:12.614174 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:12.614206 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:12.614302 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:12.614307 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:12.615846 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:12.615852 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:12.617231 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:12.617240 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:12.618341 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:12.618345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:12.619268 (+ 923us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:12.619309 (+ 41us) client_negotiation.cc:769] Sending connection context
0810 18:48:12.619405 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 18:48:12.619473 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"mutex_wait_us":852,"spinlock_wait_cycles":14208,"thread_start_us":71,"threads_started":1}
I20260810 18:48:12.619972 20281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:12.613091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.131:45545 (local address 127.0.0.1:46308)
0810 18:48:12.614148 (+ 1057us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:12.614162 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:12.614251 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:12.615219 (+ 968us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:12.615222 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:12.615240 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:12.615331 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:12.615336 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:12.616646 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:12.616649 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:12.617686 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:12.617694 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:12.619447 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:12.619454 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:12.619574 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:12.619613 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 18:48:12.619699 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 18:48:12.619787 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1001,"thread_start_us":88,"threads_started":1}
I20260810 18:48:12.620405 20282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:12.613603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38052 (local address 127.18.168.129:39263)
0810 18:48:12.614092 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:12.614099 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:12.614121 (+ 22us) server_negotiation.cc:407] Connection header received
0810 18:48:12.614213 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:12.614219 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:12.614397 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:12.614567 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:12.614784 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:12.615796 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:12.617420 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:12.618191 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:12.619962 (+ 1771us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:12.620119 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:12.620173 (+ 54us) server_negotiation.cc:299] Negotiation successful
0810 18:48:12.620266 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":137,"threads_started":1}
I20260810 18:48:12.620549 20283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:12.613734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46308 (local address 127.18.168.131:45545)
0810 18:48:12.614601 (+ 867us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:12.614611 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:12.614630 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:48:12.614699 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:12.614703 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:12.614875 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:12.615228 (+ 353us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:12.615424 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:12.616488 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:12.617805 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:12.619015 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:12.619173 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:12.619371 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:12.620230 (+ 859us) server_negotiation.cc:299] Negotiation successful
0810 18:48:12.620343 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":765,"thread_start_us":117,"threads_started":1}
I20260810 18:48:12.621613 20280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:12.612531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48998 (local address 127.18.168.130:36143)
0810 18:48:12.612888 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:12.612898 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:12.613418 (+ 520us) server_negotiation.cc:407] Connection header received
0810 18:48:12.613565 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:12.613571 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:12.613779 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:12.613961 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:12.614424 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:12.615700 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:12.617359 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:12.618191 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:12.618418 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:12.618561 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:12.621308 (+ 2747us) server_negotiation.cc:299] Negotiation successful
0810 18:48:12.621406 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":127,"threads_started":1}
I20260810 18:48:12.623200 19887 tablet_service.cc:1467] Processing CreateTablet for tablet 0ddb3cc6d66d41c491d04cc3e62a512e (DEFAULT_TABLE table=test-table [id=5d7aa55327074771812e633a029927e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:12.623365 20192 tablet_service.cc:1467] Processing CreateTablet for tablet 0ddb3cc6d66d41c491d04cc3e62a512e (DEFAULT_TABLE table=test-table [id=5d7aa55327074771812e633a029927e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:12.624203 19887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ddb3cc6d66d41c491d04cc3e62a512e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:12.624522 20192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ddb3cc6d66d41c491d04cc3e62a512e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:12.628664 20043 tablet_service.cc:1467] Processing CreateTablet for tablet 0ddb3cc6d66d41c491d04cc3e62a512e (DEFAULT_TABLE table=test-table [id=5d7aa55327074771812e633a029927e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:12.629951 20043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ddb3cc6d66d41c491d04cc3e62a512e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:12.632719 20284 tablet_bootstrap.cc:492] T 0ddb3cc6d66d41c491d04cc3e62a512e P 7c74e5c9f43e4e3d8518a83d43004c0a: Bootstrap starting.
I20260810 18:48:12.635046 20285 tablet_bootstrap.cc:492] T 0ddb3cc6d66d41c491d04cc3e62a512e P 2bd84beea3544d6cadda08601474d294: Bootstrap starting.
I20260810 18:48:12.636546 20284 tablet_bootstrap.cc:654] T 0ddb3cc6d66d41c491d04cc3e62a512e P 7c74e5c9f43e4e3d8518a83d43004c0a: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:12.638450 20284 log.cc:824] T 0ddb3cc6d66d41c491d04cc3e62a512e P 7c74e5c9f43e4e3d8518a83d43004c0a: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:12.638633 20285 tablet_bootstrap.cc:654] T 0ddb3cc6d66d41c491d04cc3e62a512e P 2bd84beea3544d6cadda08601474d294: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:12.638845 20286 tablet_bootstrap.cc:492] T 0ddb3cc6d66d41c491d04cc3e62a512e P 82530ed3297e4277b4396d158b7c07d6: Bootstrap starting.
I20260810 18:48:12.640690 20285 log.cc:824] T 0ddb3cc6d66d41c491d04cc3e62a512e P 2bd84beea3544d6cadda08601474d294: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:12.641678 20284 tablet_bootstrap.cc:492] T 0ddb3cc6d66d41c491d04cc3e62a512e P 7c74e5c9f43e4e3d8518a83d43004c0a: No bootstrap required, opened a new log
I20260810 18:48:12.641877 20284 ts_tablet_manager.cc:1397] T 0ddb3cc6d66d41c491d04cc3e62a512e P 7c74e5c9f43e4e3d8518a83d43004c0a: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260810 18:48:12.642782 20285 tablet_bootstrap.cc:492] T 0ddb3cc6d66d41c491d04cc3e62a512e P 2bd84beea3544d6cadda08601474d294: No bootstrap required, opened a new log
I20260810 18:48:12.643024 20285 ts_tablet_manager.cc:1397] T 0ddb3cc6d66d41c491d04cc3e62a512e P 2bd84beea3544d6cadda08601474d294: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260810 18:48:12.643209 20286 tablet_bootstrap.cc:654] T 0ddb3cc6d66d41c491d04cc3e62a512e P 82530ed3297e4277b4396d158b7c07d6: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:12.644958 20286 log.cc:824] T 0ddb3cc6d66d41c491d04cc3e62a512e P 82530ed3297e4277b4396d158b7c07d6: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:12.647495 20286 tablet_bootstrap.cc:492] T 0ddb3cc6d66d41c491d04cc3e62a512e P 82530ed3297e4277b4396d158b7c07d6: No bootstrap required, opened a new log
I20260810 18:48:12.647694 20286 ts_tablet_manager.cc:1397] T 0ddb3cc6d66d41c491d04cc3e62a512e P 82530ed3297e4277b4396d158b7c07d6: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260810 18:48:12.648643 20285 raft_consensus.cc:348] T 0ddb3cc6d66d41c491d04cc3e62a512e P 2bd84beea3544d6cadda08601474d294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82530ed3297e4277b4396d158b7c07d6" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36143 } } peers { permanent_uuid: "7c74e5c9f43e4e3d8518a83d43004c0a" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39263 } } peers { permanent_uuid: "2bd84beea3544d6cadda08601474d294" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45545 } }
I20260810 18:48:12.648854 20284 raft_consensus.cc:348] T 0ddb3cc6d66d41c491d04cc3e62a512e P 7c74e5c9f43e4e3d8518a83d43004c0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82530ed3297e4277b4396d158b7c07d6" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36143 } } peers { permanent_uuid: "7c74e5c9f43e4e3d8518a83d43004c0a" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39263 } } peers { permanent_uuid: "2bd84beea3544d6cadda08601474d294" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45545 } }
I20260810 18:48:12.649098 20285 raft_consensus.cc:374] T 0ddb3cc6d66d41c491d04cc3e62a512e P 2bd84beea3544d6cadda08601474d294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:12.649191 20284 raft_consensus.cc:374] T 0ddb3cc6d66d41c491d04cc3e62a512e P 7c74e5c9f43e4e3d8518a83d43004c0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:12.649189 20285 raft_consensus.cc:729] T 0ddb3cc6d66d41c491d04cc3e62a512e P 2bd84beea3544d6cadda08601474d294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bd84beea3544d6cadda08601474d294, State: Initialized, Role: FOLLOWER
I20260810 18:48:12.649312 20284 raft_consensus.cc:729] T 0ddb3cc6d66d41c491d04cc3e62a512e P 7c74e5c9f43e4e3d8518a83d43004c0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c74e5c9f43e4e3d8518a83d43004c0a, State: Initialized, Role: FOLLOWER
I20260810 18:48:12.649843 20285 consensus_queue.cc:260] T 0ddb3cc6d66d41c491d04cc3e62a512e P 2bd84beea3544d6cadda08601474d294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82530ed3297e4277b4396d158b7c07d6" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36143 } } peers { permanent_uuid: "7c74e5c9f43e4e3d8518a83d43004c0a" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39263 } } peers { permanent_uuid: "2bd84beea3544d6cadda08601474d294" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45545 } }
I20260810 18:48:12.649896 20284 consensus_queue.cc:260] T 0ddb3cc6d66d41c491d04cc3e62a512e P 7c74e5c9f43e4e3d8518a83d43004c0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82530ed3297e4277b4396d158b7c07d6" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36143 } } peers { permanent_uuid: "7c74e5c9f43e4e3d8518a83d43004c0a" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39263 } } peers { permanent_uuid: "2bd84beea3544d6cadda08601474d294" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45545 } }
I20260810 18:48:12.650796 19968 heartbeater.cc:503] Master 127.18.168.190:42239 was elected leader, sending a full tablet report...
I20260810 18:48:12.651117 20284 ts_tablet_manager.cc:1428] T 0ddb3cc6d66d41c491d04cc3e62a512e P 7c74e5c9f43e4e3d8518a83d43004c0a: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s
I20260810 18:48:12.652014 20285 ts_tablet_manager.cc:1428] T 0ddb3cc6d66d41c491d04cc3e62a512e P 2bd84beea3544d6cadda08601474d294: Time spent starting tablet: real 0.009s user 0.008s sys 0.001s
I20260810 18:48:12.652423 20264 heartbeater.cc:503] Master 127.18.168.190:42239 was elected leader, sending a full tablet report...
I20260810 18:48:12.653419 20286 raft_consensus.cc:348] T 0ddb3cc6d66d41c491d04cc3e62a512e P 82530ed3297e4277b4396d158b7c07d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82530ed3297e4277b4396d158b7c07d6" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36143 } } peers { permanent_uuid: "7c74e5c9f43e4e3d8518a83d43004c0a" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39263 } } peers { permanent_uuid: "2bd84beea3544d6cadda08601474d294" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45545 } }
I20260810 18:48:12.653774 20286 raft_consensus.cc:374] T 0ddb3cc6d66d41c491d04cc3e62a512e P 82530ed3297e4277b4396d158b7c07d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:12.653836 20286 raft_consensus.cc:729] T 0ddb3cc6d66d41c491d04cc3e62a512e P 82530ed3297e4277b4396d158b7c07d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82530ed3297e4277b4396d158b7c07d6, State: Initialized, Role: FOLLOWER
I20260810 18:48:12.654263 20286 consensus_queue.cc:260] T 0ddb3cc6d66d41c491d04cc3e62a512e P 82530ed3297e4277b4396d158b7c07d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82530ed3297e4277b4396d158b7c07d6" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36143 } } peers { permanent_uuid: "7c74e5c9f43e4e3d8518a83d43004c0a" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39263 } } peers { permanent_uuid: "2bd84beea3544d6cadda08601474d294" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45545 } }
I20260810 18:48:12.655004 20115 heartbeater.cc:503] Master 127.18.168.190:42239 was elected leader, sending a full tablet report...
I20260810 18:48:12.655260 20286 ts_tablet_manager.cc:1428] T 0ddb3cc6d66d41c491d04cc3e62a512e P 82530ed3297e4277b4396d158b7c07d6: Time spent starting tablet: real 0.007s user 0.004s sys 0.005s
W20260810 18:48:12.797755 20116 tablet.cc:2369] T 0ddb3cc6d66d41c491d04cc3e62a512e P 82530ed3297e4277b4396d158b7c07d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:48:12.816111 20265 tablet.cc:2369] T 0ddb3cc6d66d41c491d04cc3e62a512e P 2bd84beea3544d6cadda08601474d294: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:48:12.825459 19969 tablet.cc:2369] T 0ddb3cc6d66d41c491d04cc3e62a512e P 7c74e5c9f43e4e3d8518a83d43004c0a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:48:12.876837 20291 raft_consensus.cc:482] T 0ddb3cc6d66d41c491d04cc3e62a512e P 7c74e5c9f43e4e3d8518a83d43004c0a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:48:12.877223 20291 raft_consensus.cc:504] T 0ddb3cc6d66d41c491d04cc3e62a512e P 7c74e5c9f43e4e3d8518a83d43004c0a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82530ed3297e4277b4396d158b7c07d6" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36143 } } peers { permanent_uuid: "7c74e5c9f43e4e3d8518a83d43004c0a" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39263 } } peers { permanent_uuid: "2bd84beea3544d6cadda08601474d294" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45545 } }
I20260810 18:48:12.878449 20291 leader_election.cc:288] T 0ddb3cc6d66d41c491d04cc3e62a512e P 7c74e5c9f43e4e3d8518a83d43004c0a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 82530ed3297e4277b4396d158b7c07d6 (127.18.168.130:36143), 2bd84beea3544d6cadda08601474d294 (127.18.168.131:45545)
I20260810 18:48:12.882771 20293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:12.878835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:36143 (local address 127.18.168.129:46379)
0810 18:48:12.879350 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:12.879366 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:12.879660 (+ 294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:12.879912 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:12.879914 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:12.879929 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:12.880032 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:12.880036 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:12.880841 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:12.880849 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:12.881628 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:12.881638 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:12.882346 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:12.882350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:12.882458 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:12.882491 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 18:48:12.882564 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 18:48:12.882644 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":159,"threads_started":1}
I20260810 18:48:12.882823 20280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:12.878850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:46379 (local address 127.18.168.130:36143)
0810 18:48:12.879023 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:12.879028 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:12.879632 (+ 604us) server_negotiation.cc:407] Connection header received
0810 18:48:12.879743 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:12.879746 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:12.879812 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:12.879892 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:12.880140 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:12.880660 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:12.881752 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:12.882227 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:12.882319 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:12.882358 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:12.882595 (+ 237us) server_negotiation.cc:299] Negotiation successful
0810 18:48:12.882693 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 18:48:12.883093 20294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:12.878792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.131:45545 (local address 127.18.168.129:36865)
0810 18:48:12.879199 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:12.879212 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:12.879341 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:12.879710 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:12.879714 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:12.879733 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:12.880023 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:12.880037 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:12.880807 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:12.880813 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:12.881812 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:12.881822 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:12.882515 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:12.882518 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:12.882727 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:12.882752 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 18:48:12.882833 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 18:48:12.882909 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":137,"threads_started":1}
I20260810 18:48:12.883103 20283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:12.878959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:36865 (local address 127.18.168.131:45545)
0810 18:48:12.879083 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:12.879089 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:12.879328 (+ 239us) server_negotiation.cc:407] Connection header received
0810 18:48:12.879452 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:12.879466 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:12.879583 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:12.879671 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:12.880158 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:12.880667 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:12.881954 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:12.882392 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:12.882507 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:12.882579 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:12.882879 (+ 300us) server_negotiation.cc:299] Negotiation successful
0810 18:48:12.882969 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 18:48:12.883733 20065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ddb3cc6d66d41c491d04cc3e62a512e" candidate_uuid: "7c74e5c9f43e4e3d8518a83d43004c0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82530ed3297e4277b4396d158b7c07d6" is_pre_election: true
I20260810 18:48:12.883924 20213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ddb3cc6d66d41c491d04cc3e62a512e" candidate_uuid: "7c74e5c9f43e4e3d8518a83d43004c0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bd84beea3544d6cadda08601474d294" is_pre_election: true
I20260810 18:48:12.884090 20065 raft_consensus.cc:2454] T 0ddb3cc6d66d41c491d04cc3e62a512e P 82530ed3297e4277b4396d158b7c07d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c74e5c9f43e4e3d8518a83d43004c0a in term 0.
I20260810 18:48:12.884179 20213 raft_consensus.cc:2454] T 0ddb3cc6d66d41c491d04cc3e62a512e P 2bd84beea3544d6cadda08601474d294 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c74e5c9f43e4e3d8518a83d43004c0a in term 0.
I20260810 18:48:12.884624 19830 leader_election.cc:302] T 0ddb3cc6d66d41c491d04cc3e62a512e P 7c74e5c9f43e4e3d8518a83d43004c0a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2bd84beea3544d6cadda08601474d294, 7c74e5c9f43e4e3d8518a83d43004c0a, 82530ed3297e4277b4396d158b7c07d6; no voters:
I20260810 18:48:12.884912 20291 raft_consensus.cc:2781] T 0ddb3cc6d66d41c491d04cc3e62a512e P 7c74e5c9f43e4e3d8518a83d43004c0a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:48:12.885007 20291 raft_consensus.cc:482] T 0ddb3cc6d66d41c491d04cc3e62a512e P 7c74e5c9f43e4e3d8518a83d43004c0a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:48:12.885062 20291 raft_consensus.cc:3037] T 0ddb3cc6d66d41c491d04cc3e62a512e P 7c74e5c9f43e4e3d8518a83d43004c0a [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:12.886103 20291 raft_consensus.cc:504] T 0ddb3cc6d66d41c491d04cc3e62a512e P 7c74e5c9f43e4e3d8518a83d43004c0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82530ed3297e4277b4396d158b7c07d6" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36143 } } peers { permanent_uuid: "7c74e5c9f43e4e3d8518a83d43004c0a" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39263 } } peers { permanent_uuid: "2bd84beea3544d6cadda08601474d294" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45545 } }
I20260810 18:48:12.886545 20291 leader_election.cc:288] T 0ddb3cc6d66d41c491d04cc3e62a512e P 7c74e5c9f43e4e3d8518a83d43004c0a [CANDIDATE]: Term 1 election: Requested vote from peers 82530ed3297e4277b4396d158b7c07d6 (127.18.168.130:36143), 2bd84beea3544d6cadda08601474d294 (127.18.168.131:45545)
I20260810 18:48:12.886838 20065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ddb3cc6d66d41c491d04cc3e62a512e" candidate_uuid: "7c74e5c9f43e4e3d8518a83d43004c0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82530ed3297e4277b4396d158b7c07d6"
I20260810 18:48:12.886868 20213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ddb3cc6d66d41c491d04cc3e62a512e" candidate_uuid: "7c74e5c9f43e4e3d8518a83d43004c0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bd84beea3544d6cadda08601474d294"
I20260810 18:48:12.886940 20065 raft_consensus.cc:3037] T 0ddb3cc6d66d41c491d04cc3e62a512e P 82530ed3297e4277b4396d158b7c07d6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:12.886999 20213 raft_consensus.cc:3037] T 0ddb3cc6d66d41c491d04cc3e62a512e P 2bd84beea3544d6cadda08601474d294 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:12.888079 20065 raft_consensus.cc:2454] T 0ddb3cc6d66d41c491d04cc3e62a512e P 82530ed3297e4277b4396d158b7c07d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c74e5c9f43e4e3d8518a83d43004c0a in term 1.
I20260810 18:48:12.888387 19830 leader_election.cc:302] T 0ddb3cc6d66d41c491d04cc3e62a512e P 7c74e5c9f43e4e3d8518a83d43004c0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c74e5c9f43e4e3d8518a83d43004c0a, 82530ed3297e4277b4396d158b7c07d6; no voters:
I20260810 18:48:12.888550 20291 raft_consensus.cc:2781] T 0ddb3cc6d66d41c491d04cc3e62a512e P 7c74e5c9f43e4e3d8518a83d43004c0a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:12.888653 20213 raft_consensus.cc:2454] T 0ddb3cc6d66d41c491d04cc3e62a512e P 2bd84beea3544d6cadda08601474d294 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c74e5c9f43e4e3d8518a83d43004c0a in term 1.
I20260810 18:48:12.888885 20291 raft_consensus.cc:686] T 0ddb3cc6d66d41c491d04cc3e62a512e P 7c74e5c9f43e4e3d8518a83d43004c0a [term 1 LEADER]: Becoming Leader. State: Replica: 7c74e5c9f43e4e3d8518a83d43004c0a, State: Running, Role: LEADER
I20260810 18:48:12.889272 20291 consensus_queue.cc:237] T 0ddb3cc6d66d41c491d04cc3e62a512e P 7c74e5c9f43e4e3d8518a83d43004c0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82530ed3297e4277b4396d158b7c07d6" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36143 } } peers { permanent_uuid: "7c74e5c9f43e4e3d8518a83d43004c0a" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39263 } } peers { permanent_uuid: "2bd84beea3544d6cadda08601474d294" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45545 } }
I20260810 18:48:12.892736 19732 catalog_manager.cc:5495] T 0ddb3cc6d66d41c491d04cc3e62a512e P 7c74e5c9f43e4e3d8518a83d43004c0a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c74e5c9f43e4e3d8518a83d43004c0a (127.18.168.129). New cstate: current_term: 1 leader_uuid: "7c74e5c9f43e4e3d8518a83d43004c0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82530ed3297e4277b4396d158b7c07d6" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36143 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c74e5c9f43e4e3d8518a83d43004c0a" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bd84beea3544d6cadda08601474d294" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45545 } health_report { overall_health: UNKNOWN } } }
Aug 10 18:48:12 dist-test-slave-cbsf krb5kdc[19682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.168.129@KRBTEST.COM
I20260810 18:48:12.916587 20282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:12.907091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38066 (local address 127.18.168.129:39263)
0810 18:48:12.907189 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:12.907192 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:12.907515 (+ 323us) server_negotiation.cc:407] Connection header received
0810 18:48:12.907746 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:12.907749 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:12.907788 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:12.907872 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:12.908652 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:12.909221 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:12.910275 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:12.910500 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:12.910624 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:12.913324 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:12.913333 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:12.913348 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:12.913370 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:12.914936 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:12.915335 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:12.915340 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:12.915341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:12.915416 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:12.915645 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:12.915648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:12.915649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:12.915886 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:12.916037 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:12.916227 (+ 190us) server_negotiation.cc:299] Negotiation successful
0810 18:48:12.916357 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
Aug 10 18:48:12 dist-test-slave-cbsf krb5kdc[19682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387692, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 18:48:12 dist-test-slave-cbsf krb5kdc[19682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387692, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.168.129@KRBTEST.COM
I20260810 18:48:12.950554 20282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:12.936802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38080 (local address 127.18.168.129:39263)
0810 18:48:12.936937 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:12.936942 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:12.937105 (+ 163us) server_negotiation.cc:407] Connection header received
0810 18:48:12.937295 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:12.937299 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:12.937344 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:12.937434 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:12.938115 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:12.938654 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:12.939496 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:12.939730 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:12.939840 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:12.947409 (+ 7569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:12.947412 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:12.947414 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:12.947439 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:12.948825 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:12.949260 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:12.949264 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:12.949265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:12.949347 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:12.949638 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:12.949642 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:12.949644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:12.949816 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:12.949998 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:12.950267 (+ 269us) server_negotiation.cc:299] Negotiation successful
0810 18:48:12.950391 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Aug 10 18:48:12 dist-test-slave-cbsf krb5kdc[19682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 18:48:12 dist-test-slave-cbsf krb5kdc[19682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.168.129@KRBTEST.COM
I20260810 18:48:12.978935 20282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:12.969513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38082 (local address 127.18.168.129:39263)
0810 18:48:12.969697 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:12.969703 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:12.969859 (+ 156us) server_negotiation.cc:407] Connection header received
0810 18:48:12.969951 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:12.969953 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:12.970002 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:12.970086 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:12.970747 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:12.971302 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:12.972139 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:12.972377 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:12.972519 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:12.975151 (+ 2632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:12.975155 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:12.975157 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:12.975181 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:12.976681 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:12.977255 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:12.977259 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:12.977261 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:12.977348 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:12.977874 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:12.977882 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:12.977885 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:12.978093 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:12.978290 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:12.978588 (+ 298us) server_negotiation.cc:299] Negotiation successful
0810 18:48:12.978727 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260810 18:48:12.980742 19860 tablet_service.cc:2908] Rejecting scan request for tablet 0ddb3cc6d66d41c491d04cc3e62a512e: Uninitialized: clean time has not yet been initialized
I20260810 18:48:12.983242 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 19807
I20260810 18:48:12.993868 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 19973
I20260810 18:48:13.003944 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 20120
I20260810 18:48:13.015105 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 19697
2026-08-10T18:48:13Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2648 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:48:13 dist-test-slave-cbsf krb5kdc[20311](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:48:13 dist-test-slave-cbsf krb5kdc[20311](info): set up 2 sockets
Aug 10 18:48:13 dist-test-slave-cbsf krb5kdc[20311](info): commencing operation
krb5kdc: starting...
W20260810 18:48:14.145911 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:48:14 dist-test-slave-cbsf krb5kdc[20311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:48:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:48:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.168.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
I20260810 18:48:14.268362 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:42241
--webserver_interface=127.18.168.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:43671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:42241
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:14.377651 20325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:14.377913 20325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:14.377986 20325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:14.381986 20325 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:48:14.382061 20325 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 18:48:14.382091 20325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:14.382146 20325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:14.382184 20325 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:48:14.382221 20325 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:48:14.387719 20325 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:43671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:42241
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:42241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:14.388684 20325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:14.389698 20325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:14.398561 20331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:14.398566 20333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:14.398527 20330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:14.398694 20325 server_base.cc:1034] running on GCE node
I20260810 18:48:14.399250 20325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:14.400316 20325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:14.401479 20325 hybrid_clock.cc:648] HybridClock initialized: now 1786387694401461 us; error 30 us; skew 500 ppm
Aug 10 18:48:14 dist-test-slave-cbsf krb5kdc[20311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387694, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:14.404318 20325 init.cc:377] Logged in from keytab as kudu/127.18.168.190@KRBTEST.COM (short username kudu)
I20260810 18:48:14.405241 20325 webserver.cc:460] Webserver started at http://127.18.168.190:41419/ using document root <none> and password file <none>
I20260810 18:48:14.405787 20325 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:14.405848 20325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:14.406091 20325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:14.408080 20325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "93be483b2fc741eaa27083a77fca5b6d"
format_stamp: "Formatted at 2026-08-10 18:48:14 on dist-test-slave-cbsf"
server_key: "238aee0cdb97909be0edcbdde8b255b3"
server_key_iv: "1aab0582c7fcef6062763e5e1ecfb8f6"
server_key_version: "encryptionkey@0"
I20260810 18:48:14.408668 20325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "93be483b2fc741eaa27083a77fca5b6d"
format_stamp: "Formatted at 2026-08-10 18:48:14 on dist-test-slave-cbsf"
server_key: "238aee0cdb97909be0edcbdde8b255b3"
server_key_iv: "1aab0582c7fcef6062763e5e1ecfb8f6"
server_key_version: "encryptionkey@0"
I20260810 18:48:14.412263 20325 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:48:14.414556 20343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:14.415720 20325 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:48:14.415871 20325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "93be483b2fc741eaa27083a77fca5b6d"
format_stamp: "Formatted at 2026-08-10 18:48:14 on dist-test-slave-cbsf"
server_key: "238aee0cdb97909be0edcbdde8b255b3"
server_key_iv: "1aab0582c7fcef6062763e5e1ecfb8f6"
server_key_version: "encryptionkey@0"
I20260810 18:48:14.415977 20325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:14.431176 20325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:14.433845 20325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:14.434047 20325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:14.442268 20325 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:42241
I20260810 18:48:14.442296 20410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:42241 every 8 connection(s)
I20260810 18:48:14.443149 20325 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:48:14.444192 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 20325
I20260810 18:48:14.444315 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:48:14.444600 19106 external_mini_cluster.cc:1428] Setting key 09a0c426f1bdbab1cac7e1f7c2987f99
I20260810 18:48:14.446398 20411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:14.452431 20411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93be483b2fc741eaa27083a77fca5b6d: Bootstrap starting.
Aug 10 18:48:14 dist-test-slave-cbsf krb5kdc[20311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:14.454877 20411 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93be483b2fc741eaa27083a77fca5b6d: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:14.456038 20411 log.cc:824] T 00000000000000000000000000000000 P 93be483b2fc741eaa27083a77fca5b6d: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:14.458366 20411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93be483b2fc741eaa27083a77fca5b6d: No bootstrap required, opened a new log
I20260810 18:48:14.459510 20415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:14.446034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39544 (local address 127.18.168.190:42241)
0810 18:48:14.446546 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:14.446567 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:14.446601 (+ 34us) server_negotiation.cc:407] Connection header received
0810 18:48:14.447227 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:14.447242 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:14.447543 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:14.447862 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:14.448603 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:14.449657 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:14.450512 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:14.451364 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:14.451566 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:14.454491 (+ 2925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:14.454497 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:14.454510 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:14.454542 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:14.456211 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:14.456763 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:14.456768 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:14.456772 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:14.456937 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:14.457276 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:14.457280 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:14.457281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:14.457790 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:14.457993 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:14.458247 (+ 254us) server_negotiation.cc:299] Negotiation successful
0810 18:48:14.458577 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":152,"threads_started":1}
I20260810 18:48:14.461977 20411 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93be483b2fc741eaa27083a77fca5b6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93be483b2fc741eaa27083a77fca5b6d" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 42241 } }
I20260810 18:48:14.462178 20411 raft_consensus.cc:374] T 00000000000000000000000000000000 P 93be483b2fc741eaa27083a77fca5b6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:14.462221 20411 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93be483b2fc741eaa27083a77fca5b6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93be483b2fc741eaa27083a77fca5b6d, State: Initialized, Role: FOLLOWER
I20260810 18:48:14.462630 20411 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93be483b2fc741eaa27083a77fca5b6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93be483b2fc741eaa27083a77fca5b6d" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 42241 } }
I20260810 18:48:14.462788 20411 raft_consensus.cc:388] T 00000000000000000000000000000000 P 93be483b2fc741eaa27083a77fca5b6d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:48:14.462836 20411 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93be483b2fc741eaa27083a77fca5b6d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:48:14.462929 20411 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93be483b2fc741eaa27083a77fca5b6d [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:14.463739 20411 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93be483b2fc741eaa27083a77fca5b6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93be483b2fc741eaa27083a77fca5b6d" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 42241 } }
I20260810 18:48:14.464116 20411 leader_election.cc:302] T 00000000000000000000000000000000 P 93be483b2fc741eaa27083a77fca5b6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 93be483b2fc741eaa27083a77fca5b6d; no voters:
I20260810 18:48:14.464452 20411 leader_election.cc:288] T 00000000000000000000000000000000 P 93be483b2fc741eaa27083a77fca5b6d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:48:14.464542 20417 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93be483b2fc741eaa27083a77fca5b6d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:14.464843 20417 raft_consensus.cc:686] T 00000000000000000000000000000000 P 93be483b2fc741eaa27083a77fca5b6d [term 1 LEADER]: Becoming Leader. State: Replica: 93be483b2fc741eaa27083a77fca5b6d, State: Running, Role: LEADER
I20260810 18:48:14.465196 20417 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93be483b2fc741eaa27083a77fca5b6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93be483b2fc741eaa27083a77fca5b6d" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 42241 } }
I20260810 18:48:14.465780 20411 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93be483b2fc741eaa27083a77fca5b6d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:48:14.467052 20419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93be483b2fc741eaa27083a77fca5b6d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93be483b2fc741eaa27083a77fca5b6d. Latest consensus state: current_term: 1 leader_uuid: "93be483b2fc741eaa27083a77fca5b6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93be483b2fc741eaa27083a77fca5b6d" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 42241 } } }
I20260810 18:48:14.467187 20419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93be483b2fc741eaa27083a77fca5b6d [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:14.467456 20418 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93be483b2fc741eaa27083a77fca5b6d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "93be483b2fc741eaa27083a77fca5b6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93be483b2fc741eaa27083a77fca5b6d" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 42241 } } }
I20260810 18:48:14.467552 20418 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93be483b2fc741eaa27083a77fca5b6d [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:14.467558 20426 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:48:14.470753 20426 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:48:14.476316 20426 catalog_manager.cc:1347] Generated new cluster ID: 023a69772e054d4f8a9a9c487bdf95ed
I20260810 18:48:14.476378 20426 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:48:14.519613 20426 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:48:14.520999 20426 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:48:14.527004 20426 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 93be483b2fc741eaa27083a77fca5b6d: Generated new TSK 0
I20260810 18:48:14.527686 20426 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.168.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab.
Entry for principal kudu/127.18.168.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab.
I20260810 18:48:14.559918 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.129:0
--local_ip_for_outbound_sockets=127.18.168.129
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:42241
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:43671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:14.672444 20439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:14.672748 20439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:14.672843 20439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:14.676829 20439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:14.676913 20439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:14.676988 20439 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 18:48:14.677067 20439 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.129
I20260810 18:48:14.682606 20439 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:43671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.168.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.168.190:42241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20439
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.129
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:14.683727 20439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:14.684809 20439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:14.695556 20449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:14.695549 20445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:14.695590 20439 server_base.cc:1034] running on GCE node
W20260810 18:48:14.695550 20446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:14.696158 20439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:14.697007 20439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:14.698199 20439 hybrid_clock.cc:648] HybridClock initialized: now 1786387694698174 us; error 45 us; skew 500 ppm
Aug 10 18:48:14 dist-test-slave-cbsf krb5kdc[20311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387694, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:14.700958 20439 init.cc:377] Logged in from keytab as kudu/127.18.168.129@KRBTEST.COM (short username kudu)
I20260810 18:48:14.701788 20439 webserver.cc:460] Webserver started at http://127.18.168.129:40121/ using document root <none> and password file <none>
I20260810 18:48:14.702348 20439 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:14.702440 20439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:14.702663 20439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:14.704651 20439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-0/data/instance:
uuid: "6c0f1f413ecf4bb282d7815b602d32b2"
format_stamp: "Formatted at 2026-08-10 18:48:14 on dist-test-slave-cbsf"
server_key: "6cf1fc46a3fc310a4731f6be20f03ec5"
server_key_iv: "adbba802e4388a75e853b072c7c7b220"
server_key_version: "encryptionkey@0"
I20260810 18:48:14.705248 20439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance:
uuid: "6c0f1f413ecf4bb282d7815b602d32b2"
format_stamp: "Formatted at 2026-08-10 18:48:14 on dist-test-slave-cbsf"
server_key: "6cf1fc46a3fc310a4731f6be20f03ec5"
server_key_iv: "adbba802e4388a75e853b072c7c7b220"
server_key_version: "encryptionkey@0"
I20260810 18:48:14.708859 20439 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:48:14.711257 20456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:14.712414 20439 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:48:14.712546 20439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-0/wal
uuid: "6c0f1f413ecf4bb282d7815b602d32b2"
format_stamp: "Formatted at 2026-08-10 18:48:14 on dist-test-slave-cbsf"
server_key: "6cf1fc46a3fc310a4731f6be20f03ec5"
server_key_iv: "adbba802e4388a75e853b072c7c7b220"
server_key_version: "encryptionkey@0"
I20260810 18:48:14.712651 20439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:14.743395 20439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:14.746623 20439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:14.746871 20439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:14.748588 20439 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:14.748657 20439 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:14.748759 20439 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:14.748802 20439 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 18:48:14 dist-test-slave-cbsf krb5kdc[20311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387694, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:14.759675 20439 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.129:37951
I20260810 18:48:14.760007 20584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.129:37951 every 8 connection(s)
I20260810 18:48:14.760567 20439 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
I20260810 18:48:14.762930 20415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:14.749321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:36183 (local address 127.18.168.190:42241)
0810 18:48:14.749449 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:14.749453 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:14.751127 (+ 1674us) server_negotiation.cc:407] Connection header received
0810 18:48:14.752405 (+ 1278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:14.752408 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:14.752474 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:14.752587 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:14.753774 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:14.754334 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:14.755599 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:14.756018 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:14.756160 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:14.759306 (+ 3146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:14.759312 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:14.759315 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:14.759365 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:14.761121 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:14.761665 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:14.761668 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:14.761669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:14.761760 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:14.761998 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:14.762000 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:14.762001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:14.762163 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:14.762234 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:14.762650 (+ 416us) server_negotiation.cc:299] Negotiation successful
0810 18:48:14.762785 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 18:48:14.763695 20465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:14.750432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:42241 (local address 127.18.168.129:36183)
0810 18:48:14.750975 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:14.751017 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:14.752175 (+ 1158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:14.752794 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:14.752803 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:14.753180 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:14.753620 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:14.753632 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:14.754463 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:14.754467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:14.755462 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:14.755472 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:14.756196 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:14.756200 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:14.756386 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:14.757689 (+ 1303us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:14.757705 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:14.759121 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:14.761270 (+ 2149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:14.761275 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:14.761289 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:14.761564 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:14.761833 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:14.761835 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:14.761836 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:14.761946 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:14.762246 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:14.762251 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:14.762447 (+ 196us) client_negotiation.cc:769] Sending connection context
0810 18:48:14.762597 (+ 150us) client_negotiation.cc:240] Negotiation successful
0810 18:48:14.762925 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":131,"threads_started":1}
I20260810 18:48:14.765012 20586 heartbeater.cc:347] Connected to a master server at 127.18.168.190:42241
I20260810 18:48:14.765172 20586 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:14.765708 20586 heartbeater.cc:511] Master 127.18.168.190:42241 requested a full tablet report, sending...
I20260810 18:48:14.766479 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 20439
I20260810 18:48:14.766646 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance
I20260810 18:48:14.766973 19106 external_mini_cluster.cc:1428] Setting key 46dbd66c89d61b206d1bdc940ada14ef
I20260810 18:48:14.767393 20360 ts_manager.cc:194] Registered new tserver with Master: 6c0f1f413ecf4bb282d7815b602d32b2 (127.18.168.129:37951)
I20260810 18:48:14.768918 20360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.129@KRBTEST.COM'} at 127.18.168.129:36183
I20260810 18:48:14.774959 20415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:14.767419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:57529 (local address 127.18.168.190:42241)
0810 18:48:14.767535 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:14.767539 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:14.767758 (+ 219us) server_negotiation.cc:407] Connection header received
0810 18:48:14.767867 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:14.767870 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:14.767908 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:14.767996 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:14.768504 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:14.769194 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:14.770543 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:14.770815 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:14.770871 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:14.771931 (+ 1060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:14.771937 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:14.771944 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:14.771965 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:14.773349 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:14.773792 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:14.773797 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:14.773798 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:14.773882 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:14.774165 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:14.774169 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:14.774171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:14.774333 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:14.774435 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:14.774692 (+ 257us) server_negotiation.cc:299] Negotiation successful
0810 18:48:14.774794 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":47}
I20260810 18:48:14.774966 20592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:14.767292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:42241 (local address 127.18.168.129:57529)
0810 18:48:14.767661 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:14.767674 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:14.767794 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:14.768012 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:14.768014 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:14.768259 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:14.768385 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:14.768392 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:14.769347 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:14.769354 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:14.770381 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:14.770397 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:14.770985 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:14.770987 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:14.771053 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:14.771498 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:14.771508 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:14.771800 (+ 292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:14.773468 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:14.773471 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:14.773473 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:14.773675 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:14.773979 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:14.773982 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:14.773984 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:14.774069 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:14.774427 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:14.774430 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:14.774524 (+ 94us) client_negotiation.cc:769] Sending connection context
0810 18:48:14.774626 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 18:48:14.774764 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":90,"threads_started":1}
WARNING: no policy specified for kudu/127.18.168.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab.
Entry for principal kudu/127.18.168.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab.
I20260810 18:48:14.796006 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.130:0
--local_ip_for_outbound_sockets=127.18.168.130
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:42241
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:43671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:14.915535 20596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:14.915772 20596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:14.915848 20596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:14.919997 20596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:14.920049 20596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:14.920114 20596 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 18:48:14.920193 20596 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.130
I20260810 18:48:14.925333 20596 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:43671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.168.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.168.190:42241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.130
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:14.926338 20596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:14.927438 20596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:14.937096 20602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:14.937179 20601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:14.937096 20604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:14.937340 20596 server_base.cc:1034] running on GCE node
I20260810 18:48:14.937793 20596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:14.938434 20596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:14.939674 20596 hybrid_clock.cc:648] HybridClock initialized: now 1786387694939664 us; error 29 us; skew 500 ppm
Aug 10 18:48:14 dist-test-slave-cbsf krb5kdc[20311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387694, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:14.942477 20596 init.cc:377] Logged in from keytab as kudu/127.18.168.130@KRBTEST.COM (short username kudu)
I20260810 18:48:14.943307 20596 webserver.cc:460] Webserver started at http://127.18.168.130:44583/ using document root <none> and password file <none>
I20260810 18:48:14.943822 20596 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:14.943877 20596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:14.944031 20596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:14.945977 20596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-1/data/instance:
uuid: "3d581046937a435d9f19a8283579bc0a"
format_stamp: "Formatted at 2026-08-10 18:48:14 on dist-test-slave-cbsf"
server_key: "78b2d6e31c113ecc6473f4ffb7f9ac93"
server_key_iv: "b56fc9d1190631f2321c7e90cd7e0b9e"
server_key_version: "encryptionkey@0"
I20260810 18:48:14.946496 20596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance:
uuid: "3d581046937a435d9f19a8283579bc0a"
format_stamp: "Formatted at 2026-08-10 18:48:14 on dist-test-slave-cbsf"
server_key: "78b2d6e31c113ecc6473f4ffb7f9ac93"
server_key_iv: "b56fc9d1190631f2321c7e90cd7e0b9e"
server_key_version: "encryptionkey@0"
I20260810 18:48:14.950116 20596 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:48:14.952417 20613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:14.953692 20596 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:48:14.953791 20596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-1/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-1/wal
uuid: "3d581046937a435d9f19a8283579bc0a"
format_stamp: "Formatted at 2026-08-10 18:48:14 on dist-test-slave-cbsf"
server_key: "78b2d6e31c113ecc6473f4ffb7f9ac93"
server_key_iv: "b56fc9d1190631f2321c7e90cd7e0b9e"
server_key_version: "encryptionkey@0"
I20260810 18:48:14.953886 20596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:15.007848 20596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:15.011546 20596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:15.011716 20596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:15.013240 20596 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:15.013312 20596 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:15.013379 20596 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:15.013430 20596 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 18:48:15 dist-test-slave-cbsf krb5kdc[20311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387694, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:15.025084 20596 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.130:45727
I20260810 18:48:15.026185 20596 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
I20260810 18:48:15.027058 20739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.130:45727 every 8 connection(s)
I20260810 18:48:15.028839 20415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:15.014014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:41247 (local address 127.18.168.190:42241)
0810 18:48:15.014198 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:15.014204 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:15.015765 (+ 1561us) server_negotiation.cc:407] Connection header received
0810 18:48:15.016915 (+ 1150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:15.016918 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:15.016978 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:15.017056 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:15.018651 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.019273 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.021117 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.021447 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.021584 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:15.025119 (+ 3535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:15.025123 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:15.025126 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:15.025160 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:15.026688 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:15.027439 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:15.027442 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:15.027443 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:15.027519 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:15.027853 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:15.027855 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:15.027856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:15.028022 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:15.028097 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:15.028567 (+ 470us) server_negotiation.cc:299] Negotiation successful
0810 18:48:15.028669 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 18:48:15.029623 20625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:15.015034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:42241 (local address 127.18.168.130:41247)
0810 18:48:15.015557 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:15.015607 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:15.016652 (+ 1045us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:15.017268 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:15.017277 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:15.017759 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:15.018503 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:15.018526 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.019398 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.019402 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:15.020976 (+ 1574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:15.020989 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.021634 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.021638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:15.021846 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:15.023057 (+ 1211us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:15.023080 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:15.024946 (+ 1866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:15.026818 (+ 1872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:15.026824 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:15.026839 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:15.027149 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:15.027625 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:15.027628 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:15.027629 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:15.027757 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:15.028111 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:15.028115 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:15.028340 (+ 225us) client_negotiation.cc:769] Sending connection context
0810 18:48:15.028476 (+ 136us) client_negotiation.cc:240] Negotiation successful
0810 18:48:15.028776 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":163,"threads_started":1}
I20260810 18:48:15.030776 20740 heartbeater.cc:347] Connected to a master server at 127.18.168.190:42241
I20260810 18:48:15.030918 20740 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:15.031490 20740 heartbeater.cc:511] Master 127.18.168.190:42241 requested a full tablet report, sending...
I20260810 18:48:15.032621 20361 ts_manager.cc:194] Registered new tserver with Master: 3d581046937a435d9f19a8283579bc0a (127.18.168.130:45727)
I20260810 18:48:15.033384 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 20596
I20260810 18:48:15.033488 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance
I20260810 18:48:15.033435 20361 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.130@KRBTEST.COM'} at 127.18.168.130:41247
I20260810 18:48:15.033811 19106 external_mini_cluster.cc:1428] Setting key 5298fcc9363b14e64e59ded59dd386b9
I20260810 18:48:15.041697 20415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:15.033230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:41901 (local address 127.18.168.190:42241)
0810 18:48:15.033360 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:15.033365 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:15.033966 (+ 601us) server_negotiation.cc:407] Connection header received
0810 18:48:15.034080 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:15.034083 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:15.034129 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:15.034208 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:15.034777 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.035503 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.036743 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.036957 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.037041 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:15.038366 (+ 1325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:15.038369 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:15.038371 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:15.038392 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:15.039776 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:15.040331 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:15.040334 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:15.040336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:15.040467 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:15.040803 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:15.040805 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:15.040807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:15.041034 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:15.041125 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:15.041499 (+ 374us) server_negotiation.cc:299] Negotiation successful
0810 18:48:15.041592 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 18:48:15.041843 20748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:15.033115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:42241 (local address 127.18.168.130:41901)
0810 18:48:15.033874 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:15.033888 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:15.033984 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:15.034247 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:15.034251 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:15.034467 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:15.034647 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:15.034655 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.035650 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.035656 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:15.036571 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:15.036585 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.037073 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.037078 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:15.037236 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:15.037753 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:15.037770 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:15.038228 (+ 458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:15.039891 (+ 1663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:15.039897 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:15.039902 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:15.040188 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:15.040576 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:15.040579 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:15.040581 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:15.040680 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:15.041151 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:15.041157 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:15.041280 (+ 123us) client_negotiation.cc:769] Sending connection context
0810 18:48:15.041414 (+ 134us) client_negotiation.cc:240] Negotiation successful
0810 18:48:15.041582 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"thread_start_us":92,"threads_started":1}
WARNING: no policy specified for kudu/127.18.168.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab.
Entry for principal kudu/127.18.168.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab.
I20260810 18:48:15.064510 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.131:0
--local_ip_for_outbound_sockets=127.18.168.131
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:42241
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:43671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:15.184830 20752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:15.185141 20752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:15.185217 20752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:15.189327 20752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:15.189420 20752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:15.189536 20752 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 18:48:15.189620 20752 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.131
I20260810 18:48:15.194833 20752 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:43671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.168.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.168.190:42241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.131
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:15.195955 20752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:15.197017 20752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:15.206743 20757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:15.206776 20758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:15.206766 20760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:15.206813 20752 server_base.cc:1034] running on GCE node
I20260810 18:48:15.207352 20752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:15.207940 20752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:15.209120 20752 hybrid_clock.cc:648] HybridClock initialized: now 1786387695209104 us; error 31 us; skew 500 ppm
Aug 10 18:48:15 dist-test-slave-cbsf krb5kdc[20311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387695, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:15.211863 20752 init.cc:377] Logged in from keytab as kudu/127.18.168.131@KRBTEST.COM (short username kudu)
I20260810 18:48:15.212760 20752 webserver.cc:460] Webserver started at http://127.18.168.131:40695/ using document root <none> and password file <none>
I20260810 18:48:15.213318 20752 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:15.213409 20752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:15.213642 20752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:15.215677 20752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-2/data/instance:
uuid: "a324512d98564e0d9babc8f4592d090c"
format_stamp: "Formatted at 2026-08-10 18:48:15 on dist-test-slave-cbsf"
server_key: "bc8003ea1a555964de91e34db135f2f0"
server_key_iv: "8c4861d566ee1a0e3898e7331b84a555"
server_key_version: "encryptionkey@0"
I20260810 18:48:15.216291 20752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance:
uuid: "a324512d98564e0d9babc8f4592d090c"
format_stamp: "Formatted at 2026-08-10 18:48:15 on dist-test-slave-cbsf"
server_key: "bc8003ea1a555964de91e34db135f2f0"
server_key_iv: "8c4861d566ee1a0e3898e7331b84a555"
server_key_version: "encryptionkey@0"
I20260810 18:48:15.220091 20752 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:48:15.222513 20768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:15.223738 20752 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:48:15.223873 20752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-2/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-2/wal
uuid: "a324512d98564e0d9babc8f4592d090c"
format_stamp: "Formatted at 2026-08-10 18:48:15 on dist-test-slave-cbsf"
server_key: "bc8003ea1a555964de91e34db135f2f0"
server_key_iv: "8c4861d566ee1a0e3898e7331b84a555"
server_key_version: "encryptionkey@0"
I20260810 18:48:15.223980 20752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:15.255343 20752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:15.258749 20752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:15.258975 20752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:15.260385 20752 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:15.260460 20752 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:15.260540 20752 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:15.260584 20752 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 18:48:15 dist-test-slave-cbsf krb5kdc[20311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387695, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:15.277884 20415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:15.261232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:45413 (local address 127.18.168.190:42241)
0810 18:48:15.261389 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:15.261395 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:15.262389 (+ 994us) server_negotiation.cc:407] Connection header received
0810 18:48:15.264276 (+ 1887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:15.264279 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:15.264325 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:15.264380 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:15.266890 (+ 2510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.267630 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.269488 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.269748 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.269865 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:15.273719 (+ 3854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:15.273723 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:15.273726 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:15.273752 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:15.275632 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:15.276203 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:15.276207 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:15.276208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:15.276296 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:15.276633 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:15.276636 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:15.276637 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:15.276883 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:15.276990 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:15.277582 (+ 592us) server_negotiation.cc:299] Negotiation successful
0810 18:48:15.277713 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 18:48:15.278867 20782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:15.261713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:42241 (local address 127.18.168.131:45413)
0810 18:48:15.262209 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:15.262266 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:15.264019 (+ 1753us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:15.264668 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:15.264678 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:15.265427 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:15.266577 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:15.266612 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.267797 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.267814 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:15.269353 (+ 1539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:15.269363 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.269908 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.269912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:15.270124 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:15.270858 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:15.270877 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:15.273554 (+ 2677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:15.275799 (+ 2245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:15.275805 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:15.275821 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:15.276073 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:15.276389 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:15.276392 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:15.276394 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:15.276532 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:15.276990 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:15.276997 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:15.277323 (+ 326us) client_negotiation.cc:769] Sending connection context
0810 18:48:15.277521 (+ 198us) client_negotiation.cc:240] Negotiation successful
0810 18:48:15.277827 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":104,"threads_started":1}
I20260810 18:48:15.279160 20752 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.131:34283
I20260810 18:48:15.279218 20899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.131:34283 every 8 connection(s)
I20260810 18:48:15.280176 20752 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
I20260810 18:48:15.281657 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 20752
I20260810 18:48:15.281718 20900 heartbeater.cc:347] Connected to a master server at 127.18.168.190:42241
I20260810 18:48:15.281817 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance
I20260810 18:48:15.281847 20900 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:15.282146 19106 external_mini_cluster.cc:1428] Setting key 96aa29c0307f734ef4bbc9679b1fd8da
I20260810 18:48:15.282469 20900 heartbeater.cc:511] Master 127.18.168.190:42241 requested a full tablet report, sending...
I20260810 18:48:15.283670 20361 ts_manager.cc:194] Registered new tserver with Master: a324512d98564e0d9babc8f4592d090c (127.18.168.131:34283)
I20260810 18:48:15.284477 20361 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.131@KRBTEST.COM'} at 127.18.168.131:45413
I20260810 18:48:15.284513 19106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:48:15.292997 20415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:15.283274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:48403 (local address 127.18.168.190:42241)
0810 18:48:15.283395 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:15.283398 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:15.283939 (+ 541us) server_negotiation.cc:407] Connection header received
0810 18:48:15.284044 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:15.284047 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:15.284087 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:15.284166 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:15.286715 (+ 2549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.287170 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.288028 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.288401 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.288533 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:15.289678 (+ 1145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:15.289681 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:15.289686 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:15.289712 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:15.291114 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:15.291725 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:15.291730 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:15.291732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:15.291816 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:15.292154 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:15.292159 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:15.292161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:15.292333 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:15.292462 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:15.292760 (+ 298us) server_negotiation.cc:299] Negotiation successful
0810 18:48:15.292865 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 18:48:15.293051 20907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:15.283274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:42241 (local address 127.18.168.131:48403)
0810 18:48:15.283844 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:15.283860 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:15.283951 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:15.284503 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:15.284507 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:15.284875 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:15.285007 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:15.285014 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.287332 (+ 2318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.287337 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:15.287914 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:15.287922 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.288512 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.288514 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:15.288582 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:15.289176 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:15.289187 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:15.289512 (+ 325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:15.291244 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:15.291247 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:15.291250 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:15.291603 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:15.291922 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:15.291925 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:15.291927 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:15.292056 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:15.292452 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:15.292455 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:15.292561 (+ 106us) client_negotiation.cc:769] Sending connection context
0810 18:48:15.292669 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 18:48:15.292831 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":154,"threads_started":1}
Aug 10 18:48:15 dist-test-slave-cbsf krb5kdc[20311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387695, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 18:48:15 dist-test-slave-cbsf krb5kdc[20311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387695, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:15.312677 20415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:15.304773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39550 (local address 127.18.168.190:42241)
0810 18:48:15.304898 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:15.304901 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:15.304912 (+ 11us) server_negotiation.cc:407] Connection header received
0810 18:48:15.304940 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:15.304942 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:15.304974 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:15.305058 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:15.305746 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.306229 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.307082 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.307299 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.307409 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:15.309828 (+ 2419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:15.309832 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:15.309835 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:15.309866 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:15.311084 (+ 1218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:15.311508 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:15.311512 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:15.311513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:15.311592 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:15.311870 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:15.311874 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:15.311875 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:15.312042 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:15.312181 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:15.312409 (+ 228us) server_negotiation.cc:299] Negotiation successful
0810 18:48:15.312518 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 18:48:15.315380 20361 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39550:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:48:15.318753 20361 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:48:15.333263 20925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:15.327711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.131:34283 (local address 127.0.0.1:48878)
0810 18:48:15.328039 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:15.328052 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:15.328137 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:15.329040 (+ 903us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:15.329046 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:15.329065 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:15.329185 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:15.329192 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.330393 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.330396 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:15.331847 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:15.331859 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.332682 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.332686 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:15.332840 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:15.332878 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 18:48:15.332987 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 18:48:15.333117 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":123,"thread_start_us":101,"threads_started":1}
I20260810 18:48:15.333267 20926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:15.328073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48878 (local address 127.18.168.131:34283)
0810 18:48:15.328384 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:15.328395 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:15.328415 (+ 20us) server_negotiation.cc:407] Connection header received
0810 18:48:15.328488 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:15.328493 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:15.328672 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:15.329340 (+ 668us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:15.329492 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.330199 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.331957 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.332555 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.332741 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:15.332874 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:15.333036 (+ 162us) server_negotiation.cc:299] Negotiation successful
0810 18:48:15.333128 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":130,"threads_started":1}
I20260810 18:48:15.333503 20923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:15.327006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51634 (local address 127.18.168.129:37951)
0810 18:48:15.327383 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:15.327389 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:15.327675 (+ 286us) server_negotiation.cc:407] Connection header received
0810 18:48:15.327907 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:15.327914 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:15.328140 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:15.328388 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:15.329053 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.330194 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.331823 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.332603 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.332734 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:15.332864 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:15.333242 (+ 378us) server_negotiation.cc:299] Negotiation successful
0810 18:48:15.333340 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":82,"threads_started":1}
I20260810 18:48:15.333925 20922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:15.326885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:37951 (local address 127.0.0.1:51634)
0810 18:48:15.327527 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:15.327575 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:15.327751 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:15.328750 (+ 999us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:15.328756 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:15.328781 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:15.328909 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:15.328918 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.330374 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.330377 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:15.331689 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:15.331699 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.332973 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.332976 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:15.333078 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:15.333105 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 18:48:15.333707 (+ 602us) client_negotiation.cc:240] Negotiation successful
0810 18:48:15.333807 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"mutex_wait_us":26,"thread_start_us":79,"threads_started":1}
I20260810 18:48:15.334060 20924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:15.327684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56888 (local address 127.18.168.130:45727)
0810 18:48:15.328111 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:15.328123 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:15.328145 (+ 22us) server_negotiation.cc:407] Connection header received
0810 18:48:15.328250 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:15.328256 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:15.328450 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:15.329033 (+ 583us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:15.329440 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.330543 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.332118 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.332977 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.333097 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:15.333234 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:15.333840 (+ 606us) server_negotiation.cc:299] Negotiation successful
0810 18:48:15.333919 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":106,"threads_started":1}
I20260810 18:48:15.334170 20920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:15.326987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:45727 (local address 127.0.0.1:56888)
0810 18:48:15.327576 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:15.327590 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:15.327707 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:15.329184 (+ 1477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:15.329187 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:15.329203 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:15.329319 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:15.329326 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.330707 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.330710 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:15.332002 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:15.332008 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.333297 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.333300 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:15.333614 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:15.333648 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 18:48:15.333739 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 18:48:15.334046 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"mutex_wait_us":248,"thread_start_us":130,"threads_started":1}
I20260810 18:48:15.336246 20827 tablet_service.cc:1467] Processing CreateTablet for tablet e8e833e4be9a4e7abb6f1f182e90b66b (DEFAULT_TABLE table=test-table [id=145fdc47ea044da98b645cc91864920a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:15.336503 20667 tablet_service.cc:1467] Processing CreateTablet for tablet e8e833e4be9a4e7abb6f1f182e90b66b (DEFAULT_TABLE table=test-table [id=145fdc47ea044da98b645cc91864920a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:15.337361 20827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8e833e4be9a4e7abb6f1f182e90b66b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:15.337427 20667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8e833e4be9a4e7abb6f1f182e90b66b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:15.338500 20509 tablet_service.cc:1467] Processing CreateTablet for tablet e8e833e4be9a4e7abb6f1f182e90b66b (DEFAULT_TABLE table=test-table [id=145fdc47ea044da98b645cc91864920a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:15.339442 20509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8e833e4be9a4e7abb6f1f182e90b66b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:15.344151 20927 tablet_bootstrap.cc:492] T e8e833e4be9a4e7abb6f1f182e90b66b P 3d581046937a435d9f19a8283579bc0a: Bootstrap starting.
I20260810 18:48:15.345692 20928 tablet_bootstrap.cc:492] T e8e833e4be9a4e7abb6f1f182e90b66b P a324512d98564e0d9babc8f4592d090c: Bootstrap starting.
I20260810 18:48:15.346220 20929 tablet_bootstrap.cc:492] T e8e833e4be9a4e7abb6f1f182e90b66b P 6c0f1f413ecf4bb282d7815b602d32b2: Bootstrap starting.
I20260810 18:48:15.347550 20927 tablet_bootstrap.cc:654] T e8e833e4be9a4e7abb6f1f182e90b66b P 3d581046937a435d9f19a8283579bc0a: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:15.347699 20928 tablet_bootstrap.cc:654] T e8e833e4be9a4e7abb6f1f182e90b66b P a324512d98564e0d9babc8f4592d090c: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:15.348194 20929 tablet_bootstrap.cc:654] T e8e833e4be9a4e7abb6f1f182e90b66b P 6c0f1f413ecf4bb282d7815b602d32b2: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:15.348464 20927 log.cc:824] T e8e833e4be9a4e7abb6f1f182e90b66b P 3d581046937a435d9f19a8283579bc0a: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:15.348464 20928 log.cc:824] T e8e833e4be9a4e7abb6f1f182e90b66b P a324512d98564e0d9babc8f4592d090c: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:15.349015 20929 log.cc:824] T e8e833e4be9a4e7abb6f1f182e90b66b P 6c0f1f413ecf4bb282d7815b602d32b2: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:15.350519 20927 tablet_bootstrap.cc:492] T e8e833e4be9a4e7abb6f1f182e90b66b P 3d581046937a435d9f19a8283579bc0a: No bootstrap required, opened a new log
I20260810 18:48:15.350642 20928 tablet_bootstrap.cc:492] T e8e833e4be9a4e7abb6f1f182e90b66b P a324512d98564e0d9babc8f4592d090c: No bootstrap required, opened a new log
I20260810 18:48:15.350746 20927 ts_tablet_manager.cc:1397] T e8e833e4be9a4e7abb6f1f182e90b66b P 3d581046937a435d9f19a8283579bc0a: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260810 18:48:15.350828 20928 ts_tablet_manager.cc:1397] T e8e833e4be9a4e7abb6f1f182e90b66b P a324512d98564e0d9babc8f4592d090c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 18:48:15.351058 20929 tablet_bootstrap.cc:492] T e8e833e4be9a4e7abb6f1f182e90b66b P 6c0f1f413ecf4bb282d7815b602d32b2: No bootstrap required, opened a new log
I20260810 18:48:15.351212 20929 ts_tablet_manager.cc:1397] T e8e833e4be9a4e7abb6f1f182e90b66b P 6c0f1f413ecf4bb282d7815b602d32b2: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 18:48:15.354610 20927 raft_consensus.cc:348] T e8e833e4be9a4e7abb6f1f182e90b66b P 3d581046937a435d9f19a8283579bc0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37951 } } peers { permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 } } peers { permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 } }
I20260810 18:48:15.354842 20927 raft_consensus.cc:374] T e8e833e4be9a4e7abb6f1f182e90b66b P 3d581046937a435d9f19a8283579bc0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:15.354905 20927 raft_consensus.cc:729] T e8e833e4be9a4e7abb6f1f182e90b66b P 3d581046937a435d9f19a8283579bc0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d581046937a435d9f19a8283579bc0a, State: Initialized, Role: FOLLOWER
I20260810 18:48:15.355057 20928 raft_consensus.cc:348] T e8e833e4be9a4e7abb6f1f182e90b66b P a324512d98564e0d9babc8f4592d090c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37951 } } peers { permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 } } peers { permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 } }
I20260810 18:48:15.355304 20928 raft_consensus.cc:374] T e8e833e4be9a4e7abb6f1f182e90b66b P a324512d98564e0d9babc8f4592d090c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:15.355365 20928 raft_consensus.cc:729] T e8e833e4be9a4e7abb6f1f182e90b66b P a324512d98564e0d9babc8f4592d090c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a324512d98564e0d9babc8f4592d090c, State: Initialized, Role: FOLLOWER
I20260810 18:48:15.355370 20927 consensus_queue.cc:260] T e8e833e4be9a4e7abb6f1f182e90b66b P 3d581046937a435d9f19a8283579bc0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37951 } } peers { permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 } } peers { permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 } }
I20260810 18:48:15.355800 20928 consensus_queue.cc:260] T e8e833e4be9a4e7abb6f1f182e90b66b P a324512d98564e0d9babc8f4592d090c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37951 } } peers { permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 } } peers { permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 } }
I20260810 18:48:15.356114 20740 heartbeater.cc:503] Master 127.18.168.190:42241 was elected leader, sending a full tablet report...
I20260810 18:48:15.356109 20929 raft_consensus.cc:348] T e8e833e4be9a4e7abb6f1f182e90b66b P 6c0f1f413ecf4bb282d7815b602d32b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37951 } } peers { permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 } } peers { permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 } }
I20260810 18:48:15.356326 20929 raft_consensus.cc:374] T e8e833e4be9a4e7abb6f1f182e90b66b P 6c0f1f413ecf4bb282d7815b602d32b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:15.356369 20929 raft_consensus.cc:729] T e8e833e4be9a4e7abb6f1f182e90b66b P 6c0f1f413ecf4bb282d7815b602d32b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c0f1f413ecf4bb282d7815b602d32b2, State: Initialized, Role: FOLLOWER
I20260810 18:48:15.356417 20927 ts_tablet_manager.cc:1428] T e8e833e4be9a4e7abb6f1f182e90b66b P 3d581046937a435d9f19a8283579bc0a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 18:48:15.356771 20928 ts_tablet_manager.cc:1428] T e8e833e4be9a4e7abb6f1f182e90b66b P a324512d98564e0d9babc8f4592d090c: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260810 18:48:15.356813 20929 consensus_queue.cc:260] T e8e833e4be9a4e7abb6f1f182e90b66b P 6c0f1f413ecf4bb282d7815b602d32b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37951 } } peers { permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 } } peers { permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 } }
I20260810 18:48:15.357110 20900 heartbeater.cc:503] Master 127.18.168.190:42241 was elected leader, sending a full tablet report...
I20260810 18:48:15.357669 20929 ts_tablet_manager.cc:1428] T e8e833e4be9a4e7abb6f1f182e90b66b P 6c0f1f413ecf4bb282d7815b602d32b2: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260810 18:48:15.357985 20586 heartbeater.cc:503] Master 127.18.168.190:42241 was elected leader, sending a full tablet report...
I20260810 18:48:15.368299 20935 raft_consensus.cc:482] T e8e833e4be9a4e7abb6f1f182e90b66b P 3d581046937a435d9f19a8283579bc0a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:48:15.368466 20935 raft_consensus.cc:504] T e8e833e4be9a4e7abb6f1f182e90b66b P 3d581046937a435d9f19a8283579bc0a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37951 } } peers { permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 } } peers { permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 } }
I20260810 18:48:15.369503 20935 leader_election.cc:288] T e8e833e4be9a4e7abb6f1f182e90b66b P 3d581046937a435d9f19a8283579bc0a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c0f1f413ecf4bb282d7815b602d32b2 (127.18.168.129:37951), a324512d98564e0d9babc8f4592d090c (127.18.168.131:34283)
I20260810 18:48:15.373456 20625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:15.369726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:37951 (local address 127.18.168.130:43337)
0810 18:48:15.369900 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:15.369916 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:15.370013 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:15.370265 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:15.370270 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:15.370288 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:15.370509 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:15.370518 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.371373 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.371375 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:15.372358 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:15.372368 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.373139 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.373142 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:15.373234 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:15.373254 (+ 20us) client_negotiation.cc:769] Sending connection context
0810 18:48:15.373313 (+ 59us) client_negotiation.cc:240] Negotiation successful
0810 18:48:15.373364 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98}
I20260810 18:48:15.373526 20923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:15.369764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:43337 (local address 127.18.168.129:37951)
0810 18:48:15.369908 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:15.369915 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:15.369963 (+ 48us) server_negotiation.cc:407] Connection header received
0810 18:48:15.370123 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:15.370126 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:15.370163 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:15.370225 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:15.370634 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.371267 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.372496 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.373032 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.373149 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:15.373198 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:15.373349 (+ 151us) server_negotiation.cc:299] Negotiation successful
0810 18:48:15.373416 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 18:48:15.374161 20940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:15.369748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.131:34283 (local address 127.18.168.130:47585)
0810 18:48:15.370239 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:15.370252 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:15.370575 (+ 323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:15.370818 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:15.370820 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:15.370833 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:15.371167 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:15.371172 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.371915 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.371919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:15.372920 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:15.372928 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.373799 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.373801 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:15.373876 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:15.373892 (+ 16us) client_negotiation.cc:769] Sending connection context
0810 18:48:15.373949 (+ 57us) client_negotiation.cc:240] Negotiation successful
0810 18:48:15.373995 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":121,"threads_started":1}
I20260810 18:48:15.374174 20926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:15.369934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:47585 (local address 127.18.168.131:34283)
0810 18:48:15.370045 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:15.370051 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:15.370552 (+ 501us) server_negotiation.cc:407] Connection header received
0810 18:48:15.370681 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:15.370684 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:15.370728 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:15.370796 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:15.371286 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.371797 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.373032 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.373589 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.373703 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:15.373737 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:15.373978 (+ 241us) server_negotiation.cc:299] Negotiation successful
0810 18:48:15.374057 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 18:48:15.374352 20530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8e833e4be9a4e7abb6f1f182e90b66b" candidate_uuid: "3d581046937a435d9f19a8283579bc0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" is_pre_election: true
I20260810 18:48:15.374607 20530 raft_consensus.cc:2454] T e8e833e4be9a4e7abb6f1f182e90b66b P 6c0f1f413ecf4bb282d7815b602d32b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d581046937a435d9f19a8283579bc0a in term 0.
I20260810 18:48:15.375267 20616 leader_election.cc:302] T e8e833e4be9a4e7abb6f1f182e90b66b P 3d581046937a435d9f19a8283579bc0a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d581046937a435d9f19a8283579bc0a, 6c0f1f413ecf4bb282d7815b602d32b2; no voters:
I20260810 18:48:15.375540 20935 raft_consensus.cc:2781] T e8e833e4be9a4e7abb6f1f182e90b66b P 3d581046937a435d9f19a8283579bc0a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:48:15.375532 20849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8e833e4be9a4e7abb6f1f182e90b66b" candidate_uuid: "3d581046937a435d9f19a8283579bc0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a324512d98564e0d9babc8f4592d090c" is_pre_election: true
I20260810 18:48:15.375633 20935 raft_consensus.cc:482] T e8e833e4be9a4e7abb6f1f182e90b66b P 3d581046937a435d9f19a8283579bc0a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:48:15.375680 20935 raft_consensus.cc:3037] T e8e833e4be9a4e7abb6f1f182e90b66b P 3d581046937a435d9f19a8283579bc0a [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:15.375785 20849 raft_consensus.cc:2454] T e8e833e4be9a4e7abb6f1f182e90b66b P a324512d98564e0d9babc8f4592d090c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d581046937a435d9f19a8283579bc0a in term 0.
I20260810 18:48:15.376900 20935 raft_consensus.cc:504] T e8e833e4be9a4e7abb6f1f182e90b66b P 3d581046937a435d9f19a8283579bc0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37951 } } peers { permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 } } peers { permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 } }
I20260810 18:48:15.377218 20935 leader_election.cc:288] T e8e833e4be9a4e7abb6f1f182e90b66b P 3d581046937a435d9f19a8283579bc0a [CANDIDATE]: Term 1 election: Requested vote from peers 6c0f1f413ecf4bb282d7815b602d32b2 (127.18.168.129:37951), a324512d98564e0d9babc8f4592d090c (127.18.168.131:34283)
I20260810 18:48:15.377534 20530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8e833e4be9a4e7abb6f1f182e90b66b" candidate_uuid: "3d581046937a435d9f19a8283579bc0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c0f1f413ecf4bb282d7815b602d32b2"
I20260810 18:48:15.377547 20849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8e833e4be9a4e7abb6f1f182e90b66b" candidate_uuid: "3d581046937a435d9f19a8283579bc0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a324512d98564e0d9babc8f4592d090c"
I20260810 18:48:15.377630 20530 raft_consensus.cc:3037] T e8e833e4be9a4e7abb6f1f182e90b66b P 6c0f1f413ecf4bb282d7815b602d32b2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:15.377674 20849 raft_consensus.cc:3037] T e8e833e4be9a4e7abb6f1f182e90b66b P a324512d98564e0d9babc8f4592d090c [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:15.378875 20530 raft_consensus.cc:2454] T e8e833e4be9a4e7abb6f1f182e90b66b P 6c0f1f413ecf4bb282d7815b602d32b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d581046937a435d9f19a8283579bc0a in term 1.
I20260810 18:48:15.379143 20849 raft_consensus.cc:2454] T e8e833e4be9a4e7abb6f1f182e90b66b P a324512d98564e0d9babc8f4592d090c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d581046937a435d9f19a8283579bc0a in term 1.
I20260810 18:48:15.379168 20616 leader_election.cc:302] T e8e833e4be9a4e7abb6f1f182e90b66b P 3d581046937a435d9f19a8283579bc0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d581046937a435d9f19a8283579bc0a, 6c0f1f413ecf4bb282d7815b602d32b2; no voters:
I20260810 18:48:15.379329 20935 raft_consensus.cc:2781] T e8e833e4be9a4e7abb6f1f182e90b66b P 3d581046937a435d9f19a8283579bc0a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:15.379576 20935 raft_consensus.cc:686] T e8e833e4be9a4e7abb6f1f182e90b66b P 3d581046937a435d9f19a8283579bc0a [term 1 LEADER]: Becoming Leader. State: Replica: 3d581046937a435d9f19a8283579bc0a, State: Running, Role: LEADER
I20260810 18:48:15.379889 20935 consensus_queue.cc:237] T e8e833e4be9a4e7abb6f1f182e90b66b P 3d581046937a435d9f19a8283579bc0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37951 } } peers { permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 } } peers { permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 } }
I20260810 18:48:15.382880 20359 catalog_manager.cc:5495] T e8e833e4be9a4e7abb6f1f182e90b66b P 3d581046937a435d9f19a8283579bc0a reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d581046937a435d9f19a8283579bc0a (127.18.168.130). New cstate: current_term: 1 leader_uuid: "3d581046937a435d9f19a8283579bc0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 } health_report { overall_health: UNKNOWN } } }
I20260810 18:48:15.405491 20924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:15.401540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56904 (local address 127.18.168.130:45727)
0810 18:48:15.401678 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:15.401681 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:15.401787 (+ 106us) server_negotiation.cc:407] Connection header received
0810 18:48:15.401954 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:15.401956 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:15.402004 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:15.402086 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:48:15.402313 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.402909 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.403765 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.404059 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.404205 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:15.404549 (+ 344us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:48:15.405074 (+ 525us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:48:15.405205 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:15.405298 (+ 93us) server_negotiation.cc:299] Negotiation successful
0810 18:48:15.405351 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 18:48:15.411196 20530 raft_consensus.cc:1261] T e8e833e4be9a4e7abb6f1f182e90b66b P 6c0f1f413ecf4bb282d7815b602d32b2 [term 1 FOLLOWER]: Refusing update from remote peer 3d581046937a435d9f19a8283579bc0a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:15.411477 20849 raft_consensus.cc:1261] T e8e833e4be9a4e7abb6f1f182e90b66b P a324512d98564e0d9babc8f4592d090c [term 1 FOLLOWER]: Refusing update from remote peer 3d581046937a435d9f19a8283579bc0a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:15.411921 20941 consensus_queue.cc:1035] T e8e833e4be9a4e7abb6f1f182e90b66b P 3d581046937a435d9f19a8283579bc0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:15.412086 20935 consensus_queue.cc:1035] T e8e833e4be9a4e7abb6f1f182e90b66b P 3d581046937a435d9f19a8283579bc0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:15.422308 20947 mvcc.cc:205] Tried to move back new op lower bound from 7317044000396677120 to 7317044000278659072. Current Snapshot: MvccSnapshot[applied={T|T < 7317044000396677120}]
I20260810 18:48:15.432964 20361 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39550:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:15.433144 20361 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39550:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:15.435560 20361 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 93be483b2fc741eaa27083a77fca5b6d: Sending DeleteTablet for 3 replicas of tablet e8e833e4be9a4e7abb6f1f182e90b66b
I20260810 18:48:15.436299 20509 tablet_service.cc:1514] Processing DeleteTablet for tablet e8e833e4be9a4e7abb6f1f182e90b66b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:15 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:51634
I20260810 18:48:15.436321 20667 tablet_service.cc:1514] Processing DeleteTablet for tablet e8e833e4be9a4e7abb6f1f182e90b66b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:15 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:56888
I20260810 18:48:15.436837 20827 tablet_service.cc:1514] Processing DeleteTablet for tablet e8e833e4be9a4e7abb6f1f182e90b66b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:15 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:48878
I20260810 18:48:15.436924 20955 tablet_replica.cc:330] T e8e833e4be9a4e7abb6f1f182e90b66b P 3d581046937a435d9f19a8283579bc0a: stopping tablet replica
I20260810 18:48:15.437321 20955 raft_consensus.cc:2229] T e8e833e4be9a4e7abb6f1f182e90b66b P 3d581046937a435d9f19a8283579bc0a [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:48:15.437500 20954 tablet_replica.cc:330] T e8e833e4be9a4e7abb6f1f182e90b66b P 6c0f1f413ecf4bb282d7815b602d32b2: stopping tablet replica
I20260810 18:48:15.437592 20361 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39550:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 18:48:15.437760 20955 raft_consensus.cc:2258] T e8e833e4be9a4e7abb6f1f182e90b66b P 3d581046937a435d9f19a8283579bc0a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260810 18:48:15.437983 20361 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:48:15.438035 20954 raft_consensus.cc:2229] T e8e833e4be9a4e7abb6f1f182e90b66b P 6c0f1f413ecf4bb282d7815b602d32b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:15.438321 20954 raft_consensus.cc:2258] T e8e833e4be9a4e7abb6f1f182e90b66b P 6c0f1f413ecf4bb282d7815b602d32b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:15.438917 20956 tablet_replica.cc:330] T e8e833e4be9a4e7abb6f1f182e90b66b P a324512d98564e0d9babc8f4592d090c: stopping tablet replica
I20260810 18:48:15.439195 20956 raft_consensus.cc:2229] T e8e833e4be9a4e7abb6f1f182e90b66b P a324512d98564e0d9babc8f4592d090c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:15.439481 20956 raft_consensus.cc:2258] T e8e833e4be9a4e7abb6f1f182e90b66b P a324512d98564e0d9babc8f4592d090c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:15.440065 20955 ts_tablet_manager.cc:1905] T e8e833e4be9a4e7abb6f1f182e90b66b P 3d581046937a435d9f19a8283579bc0a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:15.440831 20954 ts_tablet_manager.cc:1905] T e8e833e4be9a4e7abb6f1f182e90b66b P 6c0f1f413ecf4bb282d7815b602d32b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:15.443152 20955 ts_tablet_manager.cc:1918] T e8e833e4be9a4e7abb6f1f182e90b66b P 3d581046937a435d9f19a8283579bc0a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:48:15.443305 20955 log.cc:1196] T e8e833e4be9a4e7abb6f1f182e90b66b P 3d581046937a435d9f19a8283579bc0a: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-1/wal/wals/e8e833e4be9a4e7abb6f1f182e90b66b
I20260810 18:48:15.443993 20667 tablet_service.cc:1467] Processing CreateTablet for tablet d37bd5d2e1c943d399766c0e87f706e7 (DEFAULT_TABLE table=test-table [id=6356b2682c1348fbbadaf7985ac79dbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:15.444224 20667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d37bd5d2e1c943d399766c0e87f706e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:15.444485 20509 tablet_service.cc:1467] Processing CreateTablet for tablet d37bd5d2e1c943d399766c0e87f706e7 (DEFAULT_TABLE table=test-table [id=6356b2682c1348fbbadaf7985ac79dbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:15.444783 20509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d37bd5d2e1c943d399766c0e87f706e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:15.444775 20827 tablet_service.cc:1467] Processing CreateTablet for tablet d37bd5d2e1c943d399766c0e87f706e7 (DEFAULT_TABLE table=test-table [id=6356b2682c1348fbbadaf7985ac79dbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:15.445072 20827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d37bd5d2e1c943d399766c0e87f706e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:15.446393 20927 tablet_bootstrap.cc:492] T d37bd5d2e1c943d399766c0e87f706e7 P 3d581046937a435d9f19a8283579bc0a: Bootstrap starting.
I20260810 18:48:15.447155 20929 tablet_bootstrap.cc:492] T d37bd5d2e1c943d399766c0e87f706e7 P 6c0f1f413ecf4bb282d7815b602d32b2: Bootstrap starting.
I20260810 18:48:15.447268 20927 tablet_bootstrap.cc:654] T d37bd5d2e1c943d399766c0e87f706e7 P 3d581046937a435d9f19a8283579bc0a: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:15.448256 20929 tablet_bootstrap.cc:654] T d37bd5d2e1c943d399766c0e87f706e7 P 6c0f1f413ecf4bb282d7815b602d32b2: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:15.448513 20955 ts_tablet_manager.cc:1939] T e8e833e4be9a4e7abb6f1f182e90b66b P 3d581046937a435d9f19a8283579bc0a: Deleting consensus metadata
I20260810 18:48:15.448791 20928 tablet_bootstrap.cc:492] T d37bd5d2e1c943d399766c0e87f706e7 P a324512d98564e0d9babc8f4592d090c: Bootstrap starting.
I20260810 18:48:15.449723 20928 tablet_bootstrap.cc:654] T d37bd5d2e1c943d399766c0e87f706e7 P a324512d98564e0d9babc8f4592d090c: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:15.449720 20347 catalog_manager.cc:4841] TS 3d581046937a435d9f19a8283579bc0a (127.18.168.130:45727): tablet e8e833e4be9a4e7abb6f1f182e90b66b (table test-table [id=145fdc47ea044da98b645cc91864920a]) successfully deleted
I20260810 18:48:15.450511 20927 tablet_bootstrap.cc:492] T d37bd5d2e1c943d399766c0e87f706e7 P 3d581046937a435d9f19a8283579bc0a: No bootstrap required, opened a new log
I20260810 18:48:15.450589 20927 ts_tablet_manager.cc:1397] T d37bd5d2e1c943d399766c0e87f706e7 P 3d581046937a435d9f19a8283579bc0a: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260810 18:48:15.450970 20927 raft_consensus.cc:348] T d37bd5d2e1c943d399766c0e87f706e7 P 3d581046937a435d9f19a8283579bc0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37951 } } peers { permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 } } peers { permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 } }
I20260810 18:48:15.451087 20927 raft_consensus.cc:374] T d37bd5d2e1c943d399766c0e87f706e7 P 3d581046937a435d9f19a8283579bc0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:15.451117 20927 raft_consensus.cc:729] T d37bd5d2e1c943d399766c0e87f706e7 P 3d581046937a435d9f19a8283579bc0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d581046937a435d9f19a8283579bc0a, State: Initialized, Role: FOLLOWER
I20260810 18:48:15.451249 20927 consensus_queue.cc:260] T d37bd5d2e1c943d399766c0e87f706e7 P 3d581046937a435d9f19a8283579bc0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37951 } } peers { permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 } } peers { permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 } }
I20260810 18:48:15.451316 20928 tablet_bootstrap.cc:492] T d37bd5d2e1c943d399766c0e87f706e7 P a324512d98564e0d9babc8f4592d090c: No bootstrap required, opened a new log
I20260810 18:48:15.451438 20928 ts_tablet_manager.cc:1397] T d37bd5d2e1c943d399766c0e87f706e7 P a324512d98564e0d9babc8f4592d090c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 18:48:15.451510 20927 ts_tablet_manager.cc:1428] T d37bd5d2e1c943d399766c0e87f706e7 P 3d581046937a435d9f19a8283579bc0a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 18:48:15.451869 20928 raft_consensus.cc:348] T d37bd5d2e1c943d399766c0e87f706e7 P a324512d98564e0d9babc8f4592d090c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37951 } } peers { permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 } } peers { permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 } }
I20260810 18:48:15.452056 20928 raft_consensus.cc:374] T d37bd5d2e1c943d399766c0e87f706e7 P a324512d98564e0d9babc8f4592d090c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:15.452104 20928 raft_consensus.cc:729] T d37bd5d2e1c943d399766c0e87f706e7 P a324512d98564e0d9babc8f4592d090c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a324512d98564e0d9babc8f4592d090c, State: Initialized, Role: FOLLOWER
I20260810 18:48:15.452268 20928 consensus_queue.cc:260] T d37bd5d2e1c943d399766c0e87f706e7 P a324512d98564e0d9babc8f4592d090c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37951 } } peers { permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 } } peers { permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 } }
I20260810 18:48:15.454054 20956 ts_tablet_manager.cc:1905] T e8e833e4be9a4e7abb6f1f182e90b66b P a324512d98564e0d9babc8f4592d090c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:15.454084 20928 ts_tablet_manager.cc:1428] T d37bd5d2e1c943d399766c0e87f706e7 P a324512d98564e0d9babc8f4592d090c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 18:48:15.444485 20954 ts_tablet_manager.cc:1918] T e8e833e4be9a4e7abb6f1f182e90b66b P 6c0f1f413ecf4bb282d7815b602d32b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:48:15.454365 20954 log.cc:1196] T e8e833e4be9a4e7abb6f1f182e90b66b P 6c0f1f413ecf4bb282d7815b602d32b2: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-0/wal/wals/e8e833e4be9a4e7abb6f1f182e90b66b
I20260810 18:48:15.454725 20954 ts_tablet_manager.cc:1939] T e8e833e4be9a4e7abb6f1f182e90b66b P 6c0f1f413ecf4bb282d7815b602d32b2: Deleting consensus metadata
I20260810 18:48:15.455991 20346 catalog_manager.cc:4841] TS 6c0f1f413ecf4bb282d7815b602d32b2 (127.18.168.129:37951): tablet e8e833e4be9a4e7abb6f1f182e90b66b (table test-table [id=145fdc47ea044da98b645cc91864920a]) successfully deleted
I20260810 18:48:15.456938 20929 tablet_bootstrap.cc:492] T d37bd5d2e1c943d399766c0e87f706e7 P 6c0f1f413ecf4bb282d7815b602d32b2: No bootstrap required, opened a new log
I20260810 18:48:15.457037 20929 ts_tablet_manager.cc:1397] T d37bd5d2e1c943d399766c0e87f706e7 P 6c0f1f413ecf4bb282d7815b602d32b2: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260810 18:48:15.457791 20929 raft_consensus.cc:348] T d37bd5d2e1c943d399766c0e87f706e7 P 6c0f1f413ecf4bb282d7815b602d32b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37951 } } peers { permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 } } peers { permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 } }
I20260810 18:48:15.457930 20929 raft_consensus.cc:374] T d37bd5d2e1c943d399766c0e87f706e7 P 6c0f1f413ecf4bb282d7815b602d32b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:15.457967 20929 raft_consensus.cc:729] T d37bd5d2e1c943d399766c0e87f706e7 P 6c0f1f413ecf4bb282d7815b602d32b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c0f1f413ecf4bb282d7815b602d32b2, State: Initialized, Role: FOLLOWER
I20260810 18:48:15.458097 20929 consensus_queue.cc:260] T d37bd5d2e1c943d399766c0e87f706e7 P 6c0f1f413ecf4bb282d7815b602d32b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37951 } } peers { permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 } } peers { permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 } }
I20260810 18:48:15.458367 20929 ts_tablet_manager.cc:1428] T d37bd5d2e1c943d399766c0e87f706e7 P 6c0f1f413ecf4bb282d7815b602d32b2: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260810 18:48:15.459156 20956 ts_tablet_manager.cc:1918] T e8e833e4be9a4e7abb6f1f182e90b66b P a324512d98564e0d9babc8f4592d090c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:48:15.459256 20956 log.cc:1196] T e8e833e4be9a4e7abb6f1f182e90b66b P a324512d98564e0d9babc8f4592d090c: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-2/wal/wals/e8e833e4be9a4e7abb6f1f182e90b66b
I20260810 18:48:15.459653 20956 ts_tablet_manager.cc:1939] T e8e833e4be9a4e7abb6f1f182e90b66b P a324512d98564e0d9babc8f4592d090c: Deleting consensus metadata
I20260810 18:48:15.461027 20345 catalog_manager.cc:4841] TS a324512d98564e0d9babc8f4592d090c (127.18.168.131:34283): tablet e8e833e4be9a4e7abb6f1f182e90b66b (table test-table [id=145fdc47ea044da98b645cc91864920a]) successfully deleted
W20260810 18:48:15.512194 20587 tablet.cc:2369] T d37bd5d2e1c943d399766c0e87f706e7 P 6c0f1f413ecf4bb282d7815b602d32b2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:48:15.527526 20741 tablet.cc:2369] T d37bd5d2e1c943d399766c0e87f706e7 P 3d581046937a435d9f19a8283579bc0a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:48:15.531363 20901 tablet.cc:2369] T d37bd5d2e1c943d399766c0e87f706e7 P a324512d98564e0d9babc8f4592d090c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:48:15.665575 20937 raft_consensus.cc:482] T d37bd5d2e1c943d399766c0e87f706e7 P 6c0f1f413ecf4bb282d7815b602d32b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:48:15.665768 20937 raft_consensus.cc:504] T d37bd5d2e1c943d399766c0e87f706e7 P 6c0f1f413ecf4bb282d7815b602d32b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37951 } } peers { permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 } } peers { permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 } }
I20260810 18:48:15.667258 20937 leader_election.cc:288] T d37bd5d2e1c943d399766c0e87f706e7 P 6c0f1f413ecf4bb282d7815b602d32b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d581046937a435d9f19a8283579bc0a (127.18.168.130:45727), a324512d98564e0d9babc8f4592d090c (127.18.168.131:34283)
I20260810 18:48:15.672039 20926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:15.667688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:58121 (local address 127.18.168.131:34283)
0810 18:48:15.667861 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:15.667866 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:15.668067 (+ 201us) server_negotiation.cc:407] Connection header received
0810 18:48:15.668229 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:15.668234 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:15.668290 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:15.668359 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:15.669019 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.669549 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.670757 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.671186 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.671276 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:15.671317 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:15.671848 (+ 531us) server_negotiation.cc:299] Negotiation successful
0810 18:48:15.671915 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 18:48:15.672051 20924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:15.667692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:47703 (local address 127.18.168.130:45727)
0810 18:48:15.667962 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:15.667966 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:15.668125 (+ 159us) server_negotiation.cc:407] Connection header received
0810 18:48:15.668268 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:15.668270 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:15.668312 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:15.668400 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:15.668817 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.669398 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.670869 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.671298 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.671456 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:15.671523 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:15.671871 (+ 348us) server_negotiation.cc:299] Negotiation successful
0810 18:48:15.671946 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194}
I20260810 18:48:15.672170 20961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:15.667588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:45727 (local address 127.18.168.129:47703)
0810 18:48:15.667991 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:15.668005 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:15.668173 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:15.668436 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:15.668439 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:15.668477 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:15.668636 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:15.668642 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.669512 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.669516 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:15.670745 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:15.670756 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.671430 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.671433 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:15.671708 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:15.671735 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 18:48:15.671835 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 18:48:15.671904 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":130,"spinlock_wait_cycles":896,"thread_start_us":197,"threads_started":1}
I20260810 18:48:15.672200 20960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:15.667562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.131:34283 (local address 127.18.168.129:58121)
0810 18:48:15.667933 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:15.667949 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:15.668092 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:15.668402 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:15.668405 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:15.668421 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:15.668890 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:15.668900 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.669669 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.669671 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:15.670607 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:15.670617 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.671433 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.671439 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:15.671699 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:15.671728 (+ 29us) client_negotiation.cc:769] Sending connection context
0810 18:48:15.671792 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 18:48:15.671887 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"mutex_wait_us":87,"spinlock_wait_cycles":14592,"thread_start_us":89,"threads_started":1}
I20260810 18:48:15.672806 20849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d37bd5d2e1c943d399766c0e87f706e7" candidate_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a324512d98564e0d9babc8f4592d090c" is_pre_election: true
I20260810 18:48:15.672807 20684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d37bd5d2e1c943d399766c0e87f706e7" candidate_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d581046937a435d9f19a8283579bc0a" is_pre_election: true
I20260810 18:48:15.672961 20849 raft_consensus.cc:2454] T d37bd5d2e1c943d399766c0e87f706e7 P a324512d98564e0d9babc8f4592d090c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c0f1f413ecf4bb282d7815b602d32b2 in term 0.
I20260810 18:48:15.673019 20684 raft_consensus.cc:2454] T d37bd5d2e1c943d399766c0e87f706e7 P 3d581046937a435d9f19a8283579bc0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c0f1f413ecf4bb282d7815b602d32b2 in term 0.
I20260810 18:48:15.673471 20458 leader_election.cc:302] T d37bd5d2e1c943d399766c0e87f706e7 P 6c0f1f413ecf4bb282d7815b602d32b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c0f1f413ecf4bb282d7815b602d32b2, a324512d98564e0d9babc8f4592d090c; no voters:
I20260810 18:48:15.673715 20937 raft_consensus.cc:2781] T d37bd5d2e1c943d399766c0e87f706e7 P 6c0f1f413ecf4bb282d7815b602d32b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:48:15.673815 20937 raft_consensus.cc:482] T d37bd5d2e1c943d399766c0e87f706e7 P 6c0f1f413ecf4bb282d7815b602d32b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:48:15.673856 20937 raft_consensus.cc:3037] T d37bd5d2e1c943d399766c0e87f706e7 P 6c0f1f413ecf4bb282d7815b602d32b2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:15.674890 20937 raft_consensus.cc:504] T d37bd5d2e1c943d399766c0e87f706e7 P 6c0f1f413ecf4bb282d7815b602d32b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37951 } } peers { permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 } } peers { permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 } }
I20260810 18:48:15.675252 20937 leader_election.cc:288] T d37bd5d2e1c943d399766c0e87f706e7 P 6c0f1f413ecf4bb282d7815b602d32b2 [CANDIDATE]: Term 1 election: Requested vote from peers 3d581046937a435d9f19a8283579bc0a (127.18.168.130:45727), a324512d98564e0d9babc8f4592d090c (127.18.168.131:34283)
I20260810 18:48:15.675576 20684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d37bd5d2e1c943d399766c0e87f706e7" candidate_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d581046937a435d9f19a8283579bc0a"
I20260810 18:48:15.675660 20849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d37bd5d2e1c943d399766c0e87f706e7" candidate_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a324512d98564e0d9babc8f4592d090c"
I20260810 18:48:15.675695 20684 raft_consensus.cc:3037] T d37bd5d2e1c943d399766c0e87f706e7 P 3d581046937a435d9f19a8283579bc0a [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:15.675772 20849 raft_consensus.cc:3037] T d37bd5d2e1c943d399766c0e87f706e7 P a324512d98564e0d9babc8f4592d090c [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:15.677265 20849 raft_consensus.cc:2454] T d37bd5d2e1c943d399766c0e87f706e7 P a324512d98564e0d9babc8f4592d090c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c0f1f413ecf4bb282d7815b602d32b2 in term 1.
I20260810 18:48:15.677265 20684 raft_consensus.cc:2454] T d37bd5d2e1c943d399766c0e87f706e7 P 3d581046937a435d9f19a8283579bc0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c0f1f413ecf4bb282d7815b602d32b2 in term 1.
I20260810 18:48:15.677536 20458 leader_election.cc:302] T d37bd5d2e1c943d399766c0e87f706e7 P 6c0f1f413ecf4bb282d7815b602d32b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c0f1f413ecf4bb282d7815b602d32b2, a324512d98564e0d9babc8f4592d090c; no voters:
I20260810 18:48:15.677762 20937 raft_consensus.cc:2781] T d37bd5d2e1c943d399766c0e87f706e7 P 6c0f1f413ecf4bb282d7815b602d32b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:15.677929 20937 raft_consensus.cc:686] T d37bd5d2e1c943d399766c0e87f706e7 P 6c0f1f413ecf4bb282d7815b602d32b2 [term 1 LEADER]: Becoming Leader. State: Replica: 6c0f1f413ecf4bb282d7815b602d32b2, State: Running, Role: LEADER
I20260810 18:48:15.678185 20937 consensus_queue.cc:237] T d37bd5d2e1c943d399766c0e87f706e7 P 6c0f1f413ecf4bb282d7815b602d32b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37951 } } peers { permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 } } peers { permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 } }
I20260810 18:48:15.680513 20361 catalog_manager.cc:5495] T d37bd5d2e1c943d399766c0e87f706e7 P 6c0f1f413ecf4bb282d7815b602d32b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c0f1f413ecf4bb282d7815b602d32b2 (127.18.168.129). New cstate: current_term: 1 leader_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37951 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 } health_report { overall_health: UNKNOWN } } }
Aug 10 18:48:15 dist-test-slave-cbsf krb5kdc[20311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387694, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:15.768575 20971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:15.758460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:42241 (local address 127.0.0.1:39556)
0810 18:48:15.758795 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:15.758807 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:15.758940 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:15.759188 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:15.759191 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:15.759823 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:15.760034 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:15.760043 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.760889 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.760893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:15.761758 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:15.761769 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.762209 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.762212 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:15.762303 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:15.762893 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:15.762906 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:15.764547 (+ 1641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:15.766864 (+ 2317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:15.766876 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:15.766882 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:15.767170 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:15.767499 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:15.767504 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:15.767506 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:15.767611 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:15.767970 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:15.767978 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:15.768097 (+ 119us) client_negotiation.cc:769] Sending connection context
0810 18:48:15.768207 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 18:48:15.768359 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":78,"threads_started":1}
I20260810 18:48:15.768574 20415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:15.758507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39556 (local address 127.18.168.190:42241)
0810 18:48:15.758662 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:15.758667 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:15.758882 (+ 215us) server_negotiation.cc:407] Connection header received
0810 18:48:15.759030 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:15.759036 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:15.759085 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:15.759166 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:15.760173 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.760734 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.761875 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.762093 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.762171 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:15.764743 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:15.764753 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:15.764755 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:15.764775 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:15.766752 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:15.767275 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:15.767281 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:15.767286 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:15.767366 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:15.767695 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:15.767698 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:15.767699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:15.767864 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:15.767967 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:15.768262 (+ 295us) server_negotiation.cc:299] Negotiation successful
0810 18:48:15.768412 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":55}
I20260810 18:48:15.772682 20361 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:39556:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 18:48:15.773511 20361 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:48:15.778362 20509 tablet_service.cc:1467] Processing CreateTablet for tablet 9e867c6ca1ac44de8d4558763c678532 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cdfdef7aac9a42de857eb309914c74ce]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 18:48:15.778498 20667 tablet_service.cc:1467] Processing CreateTablet for tablet 9e867c6ca1ac44de8d4558763c678532 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cdfdef7aac9a42de857eb309914c74ce]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 18:48:15.778781 20509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e867c6ca1ac44de8d4558763c678532. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:15.778810 20667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e867c6ca1ac44de8d4558763c678532. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:15.778852 20827 tablet_service.cc:1467] Processing CreateTablet for tablet 9e867c6ca1ac44de8d4558763c678532 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cdfdef7aac9a42de857eb309914c74ce]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 18:48:15.779241 20827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e867c6ca1ac44de8d4558763c678532. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:15.781955 20928 tablet_bootstrap.cc:492] T 9e867c6ca1ac44de8d4558763c678532 P a324512d98564e0d9babc8f4592d090c: Bootstrap starting.
I20260810 18:48:15.782166 20929 tablet_bootstrap.cc:492] T 9e867c6ca1ac44de8d4558763c678532 P 6c0f1f413ecf4bb282d7815b602d32b2: Bootstrap starting.
I20260810 18:48:15.783073 20929 tablet_bootstrap.cc:654] T 9e867c6ca1ac44de8d4558763c678532 P 6c0f1f413ecf4bb282d7815b602d32b2: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:15.783190 20928 tablet_bootstrap.cc:654] T 9e867c6ca1ac44de8d4558763c678532 P a324512d98564e0d9babc8f4592d090c: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:15.784363 20927 tablet_bootstrap.cc:492] T 9e867c6ca1ac44de8d4558763c678532 P 3d581046937a435d9f19a8283579bc0a: Bootstrap starting.
I20260810 18:48:15.785043 20929 tablet_bootstrap.cc:492] T 9e867c6ca1ac44de8d4558763c678532 P 6c0f1f413ecf4bb282d7815b602d32b2: No bootstrap required, opened a new log
I20260810 18:48:15.785144 20929 ts_tablet_manager.cc:1397] T 9e867c6ca1ac44de8d4558763c678532 P 6c0f1f413ecf4bb282d7815b602d32b2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.001s
I20260810 18:48:15.785439 20927 tablet_bootstrap.cc:654] T 9e867c6ca1ac44de8d4558763c678532 P 3d581046937a435d9f19a8283579bc0a: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:15.785451 20928 tablet_bootstrap.cc:492] T 9e867c6ca1ac44de8d4558763c678532 P a324512d98564e0d9babc8f4592d090c: No bootstrap required, opened a new log
I20260810 18:48:15.785535 20928 ts_tablet_manager.cc:1397] T 9e867c6ca1ac44de8d4558763c678532 P a324512d98564e0d9babc8f4592d090c: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260810 18:48:15.785842 20929 raft_consensus.cc:348] T 9e867c6ca1ac44de8d4558763c678532 P 6c0f1f413ecf4bb282d7815b602d32b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37951 } } peers { permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 } } peers { permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 } }
I20260810 18:48:15.785964 20929 raft_consensus.cc:374] T 9e867c6ca1ac44de8d4558763c678532 P 6c0f1f413ecf4bb282d7815b602d32b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:15.786005 20929 raft_consensus.cc:729] T 9e867c6ca1ac44de8d4558763c678532 P 6c0f1f413ecf4bb282d7815b602d32b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c0f1f413ecf4bb282d7815b602d32b2, State: Initialized, Role: FOLLOWER
I20260810 18:48:15.786191 20929 consensus_queue.cc:260] T 9e867c6ca1ac44de8d4558763c678532 P 6c0f1f413ecf4bb282d7815b602d32b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37951 } } peers { permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 } } peers { permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 } }
I20260810 18:48:15.786223 20928 raft_consensus.cc:348] T 9e867c6ca1ac44de8d4558763c678532 P a324512d98564e0d9babc8f4592d090c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37951 } } peers { permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 } } peers { permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 } }
I20260810 18:48:15.786332 20928 raft_consensus.cc:374] T 9e867c6ca1ac44de8d4558763c678532 P a324512d98564e0d9babc8f4592d090c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:15.786370 20928 raft_consensus.cc:729] T 9e867c6ca1ac44de8d4558763c678532 P a324512d98564e0d9babc8f4592d090c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a324512d98564e0d9babc8f4592d090c, State: Initialized, Role: FOLLOWER
I20260810 18:48:15.786500 20937 tablet_replica.cc:439] T 9e867c6ca1ac44de8d4558763c678532 P 6c0f1f413ecf4bb282d7815b602d32b2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37951 } } peers { permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 } } peers { permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 } } }
I20260810 18:48:15.786531 20928 consensus_queue.cc:260] T 9e867c6ca1ac44de8d4558763c678532 P a324512d98564e0d9babc8f4592d090c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37951 } } peers { permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 } } peers { permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 } }
I20260810 18:48:15.786615 20937 tablet_replica.cc:442] T 9e867c6ca1ac44de8d4558763c678532 P 6c0f1f413ecf4bb282d7815b602d32b2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:48:15.786836 20929 ts_tablet_manager.cc:1428] T 9e867c6ca1ac44de8d4558763c678532 P 6c0f1f413ecf4bb282d7815b602d32b2: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260810 18:48:15.786863 20927 tablet_bootstrap.cc:492] T 9e867c6ca1ac44de8d4558763c678532 P 3d581046937a435d9f19a8283579bc0a: No bootstrap required, opened a new log
I20260810 18:48:15.786767 20936 tablet_replica.cc:439] T 9e867c6ca1ac44de8d4558763c678532 P a324512d98564e0d9babc8f4592d090c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37951 } } peers { permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 } } peers { permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 } } }
I20260810 18:48:15.786942 20936 tablet_replica.cc:442] T 9e867c6ca1ac44de8d4558763c678532 P a324512d98564e0d9babc8f4592d090c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:48:15.786937 20927 ts_tablet_manager.cc:1397] T 9e867c6ca1ac44de8d4558763c678532 P 3d581046937a435d9f19a8283579bc0a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 18:48:15.787317 20927 raft_consensus.cc:348] T 9e867c6ca1ac44de8d4558763c678532 P 3d581046937a435d9f19a8283579bc0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37951 } } peers { permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 } } peers { permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 } }
I20260810 18:48:15.787433 20927 raft_consensus.cc:374] T 9e867c6ca1ac44de8d4558763c678532 P 3d581046937a435d9f19a8283579bc0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:15.787470 20927 raft_consensus.cc:729] T 9e867c6ca1ac44de8d4558763c678532 P 3d581046937a435d9f19a8283579bc0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d581046937a435d9f19a8283579bc0a, State: Initialized, Role: FOLLOWER
I20260810 18:48:15.787602 20927 consensus_queue.cc:260] T 9e867c6ca1ac44de8d4558763c678532 P 3d581046937a435d9f19a8283579bc0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37951 } } peers { permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 } } peers { permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 } }
I20260810 18:48:15.787788 20928 ts_tablet_manager.cc:1428] T 9e867c6ca1ac44de8d4558763c678532 P a324512d98564e0d9babc8f4592d090c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260810 18:48:15.787839 20935 tablet_replica.cc:439] T 9e867c6ca1ac44de8d4558763c678532 P 3d581046937a435d9f19a8283579bc0a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37951 } } peers { permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 } } peers { permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 } } }
I20260810 18:48:15.787997 20935 tablet_replica.cc:442] T 9e867c6ca1ac44de8d4558763c678532 P 3d581046937a435d9f19a8283579bc0a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:48:15.788250 20927 ts_tablet_manager.cc:1428] T 9e867c6ca1ac44de8d4558763c678532 P 3d581046937a435d9f19a8283579bc0a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 18:48:15.841634 20937 raft_consensus.cc:482] T 9e867c6ca1ac44de8d4558763c678532 P 6c0f1f413ecf4bb282d7815b602d32b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:48:15.841784 20937 raft_consensus.cc:504] T 9e867c6ca1ac44de8d4558763c678532 P 6c0f1f413ecf4bb282d7815b602d32b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37951 } } peers { permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 } } peers { permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 } }
I20260810 18:48:15.842325 20937 leader_election.cc:288] T 9e867c6ca1ac44de8d4558763c678532 P 6c0f1f413ecf4bb282d7815b602d32b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d581046937a435d9f19a8283579bc0a (127.18.168.130:45727), a324512d98564e0d9babc8f4592d090c (127.18.168.131:34283)
I20260810 18:48:15.842789 20684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e867c6ca1ac44de8d4558763c678532" candidate_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d581046937a435d9f19a8283579bc0a" is_pre_election: true
I20260810 18:48:15.842813 20849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e867c6ca1ac44de8d4558763c678532" candidate_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a324512d98564e0d9babc8f4592d090c" is_pre_election: true
I20260810 18:48:15.842967 20684 raft_consensus.cc:2454] T 9e867c6ca1ac44de8d4558763c678532 P 3d581046937a435d9f19a8283579bc0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c0f1f413ecf4bb282d7815b602d32b2 in term 0.
I20260810 18:48:15.842972 20849 raft_consensus.cc:2454] T 9e867c6ca1ac44de8d4558763c678532 P a324512d98564e0d9babc8f4592d090c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c0f1f413ecf4bb282d7815b602d32b2 in term 0.
I20260810 18:48:15.843358 20460 leader_election.cc:302] T 9e867c6ca1ac44de8d4558763c678532 P 6c0f1f413ecf4bb282d7815b602d32b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d581046937a435d9f19a8283579bc0a, 6c0f1f413ecf4bb282d7815b602d32b2; no voters:
I20260810 18:48:15.843549 20937 raft_consensus.cc:2781] T 9e867c6ca1ac44de8d4558763c678532 P 6c0f1f413ecf4bb282d7815b602d32b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:48:15.843616 20937 raft_consensus.cc:482] T 9e867c6ca1ac44de8d4558763c678532 P 6c0f1f413ecf4bb282d7815b602d32b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:48:15.843663 20937 raft_consensus.cc:3037] T 9e867c6ca1ac44de8d4558763c678532 P 6c0f1f413ecf4bb282d7815b602d32b2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:15.844899 20937 raft_consensus.cc:504] T 9e867c6ca1ac44de8d4558763c678532 P 6c0f1f413ecf4bb282d7815b602d32b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37951 } } peers { permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 } } peers { permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 } }
I20260810 18:48:15.845278 20937 leader_election.cc:288] T 9e867c6ca1ac44de8d4558763c678532 P 6c0f1f413ecf4bb282d7815b602d32b2 [CANDIDATE]: Term 1 election: Requested vote from peers 3d581046937a435d9f19a8283579bc0a (127.18.168.130:45727), a324512d98564e0d9babc8f4592d090c (127.18.168.131:34283)
I20260810 18:48:15.845609 20684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e867c6ca1ac44de8d4558763c678532" candidate_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d581046937a435d9f19a8283579bc0a"
I20260810 18:48:15.845666 20849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e867c6ca1ac44de8d4558763c678532" candidate_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a324512d98564e0d9babc8f4592d090c"
I20260810 18:48:15.845753 20684 raft_consensus.cc:3037] T 9e867c6ca1ac44de8d4558763c678532 P 3d581046937a435d9f19a8283579bc0a [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:15.845783 20849 raft_consensus.cc:3037] T 9e867c6ca1ac44de8d4558763c678532 P a324512d98564e0d9babc8f4592d090c [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:15.846951 20849 raft_consensus.cc:2454] T 9e867c6ca1ac44de8d4558763c678532 P a324512d98564e0d9babc8f4592d090c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c0f1f413ecf4bb282d7815b602d32b2 in term 1.
I20260810 18:48:15.847177 20684 raft_consensus.cc:2454] T 9e867c6ca1ac44de8d4558763c678532 P 3d581046937a435d9f19a8283579bc0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c0f1f413ecf4bb282d7815b602d32b2 in term 1.
I20260810 18:48:15.847278 20458 leader_election.cc:302] T 9e867c6ca1ac44de8d4558763c678532 P 6c0f1f413ecf4bb282d7815b602d32b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c0f1f413ecf4bb282d7815b602d32b2, a324512d98564e0d9babc8f4592d090c; no voters:
I20260810 18:48:15.847491 20937 raft_consensus.cc:2781] T 9e867c6ca1ac44de8d4558763c678532 P 6c0f1f413ecf4bb282d7815b602d32b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:15.847609 20937 raft_consensus.cc:686] T 9e867c6ca1ac44de8d4558763c678532 P 6c0f1f413ecf4bb282d7815b602d32b2 [term 1 LEADER]: Becoming Leader. State: Replica: 6c0f1f413ecf4bb282d7815b602d32b2, State: Running, Role: LEADER
I20260810 18:48:15.847756 20937 consensus_queue.cc:237] T 9e867c6ca1ac44de8d4558763c678532 P 6c0f1f413ecf4bb282d7815b602d32b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37951 } } peers { permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 } } peers { permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 } }
I20260810 18:48:15.848608 20962 tablet_replica.cc:439] T 9e867c6ca1ac44de8d4558763c678532 P 6c0f1f413ecf4bb282d7815b602d32b2: TxnStatusTablet state changed. Reason: New leader 6c0f1f413ecf4bb282d7815b602d32b2. Latest consensus state: current_term: 1 leader_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37951 } } peers { permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 } } peers { permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 } } }
I20260810 18:48:15.848734 20962 tablet_replica.cc:442] T 9e867c6ca1ac44de8d4558763c678532 P 6c0f1f413ecf4bb282d7815b602d32b2: This TxnStatusTablet replica's current role is: LEADER
I20260810 18:48:15.849620 20976 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 18:48:15.849834 20976 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 18:48:15.849786 20359 catalog_manager.cc:5495] T 9e867c6ca1ac44de8d4558763c678532 P 6c0f1f413ecf4bb282d7815b602d32b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c0f1f413ecf4bb282d7815b602d32b2 (127.18.168.129). New cstate: current_term: 1 leader_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37951 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 } health_report { overall_health: UNKNOWN } } }
I20260810 18:48:15.873171 20971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:15.868639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:37951 (local address 127.0.0.1:51650)
0810 18:48:15.868876 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:15.868891 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:15.869007 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:15.869484 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:15.869486 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:15.869493 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:48:15.869600 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:15.869605 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.870680 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.870683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:15.871322 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:15.871330 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.871953 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.871956 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:15.872034 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:15.872084 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:48:15.872895 (+ 811us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:48:15.872912 (+ 17us) client_negotiation.cc:769] Sending connection context
0810 18:48:15.872977 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 18:48:15.873056 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103}
I20260810 18:48:15.873188 20923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:15.868777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51650 (local address 127.18.168.129:37951)
0810 18:48:15.868901 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:15.868906 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:15.868944 (+ 38us) server_negotiation.cc:407] Connection header received
0810 18:48:15.869213 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:15.869219 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:15.869301 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:15.869443 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:48:15.869715 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.870560 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.871441 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.871825 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.871930 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:15.872231 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:48:15.872793 (+ 562us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:48:15.872907 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:15.872991 (+ 84us) server_negotiation.cc:299] Negotiation successful
0810 18:48:15.873053 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 18:48:15.876248 20849 raft_consensus.cc:1261] T 9e867c6ca1ac44de8d4558763c678532 P a324512d98564e0d9babc8f4592d090c [term 1 FOLLOWER]: Refusing update from remote peer 6c0f1f413ecf4bb282d7815b602d32b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:15.876449 20684 raft_consensus.cc:1261] T 9e867c6ca1ac44de8d4558763c678532 P 3d581046937a435d9f19a8283579bc0a [term 1 FOLLOWER]: Refusing update from remote peer 6c0f1f413ecf4bb282d7815b602d32b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:15.876891 20962 consensus_queue.cc:1035] T 9e867c6ca1ac44de8d4558763c678532 P 6c0f1f413ecf4bb282d7815b602d32b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:15.877117 20937 consensus_queue.cc:1035] T 9e867c6ca1ac44de8d4558763c678532 P 6c0f1f413ecf4bb282d7815b602d32b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:15.878604 20936 tablet_replica.cc:439] T 9e867c6ca1ac44de8d4558763c678532 P a324512d98564e0d9babc8f4592d090c: TxnStatusTablet state changed. Reason: New leader 6c0f1f413ecf4bb282d7815b602d32b2. Latest consensus state: current_term: 1 leader_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37951 } } peers { permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 } } peers { permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 } } }
I20260810 18:48:15.878790 20936 tablet_replica.cc:442] T 9e867c6ca1ac44de8d4558763c678532 P a324512d98564e0d9babc8f4592d090c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:48:15.878876 20935 tablet_replica.cc:439] T 9e867c6ca1ac44de8d4558763c678532 P 3d581046937a435d9f19a8283579bc0a: TxnStatusTablet state changed. Reason: New leader 6c0f1f413ecf4bb282d7815b602d32b2. Latest consensus state: current_term: 1 leader_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37951 } } peers { permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 } } peers { permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 } } }
I20260810 18:48:15.878988 20935 tablet_replica.cc:442] T 9e867c6ca1ac44de8d4558763c678532 P 3d581046937a435d9f19a8283579bc0a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:48:15.881409 20937 tablet_replica.cc:439] T 9e867c6ca1ac44de8d4558763c678532 P 6c0f1f413ecf4bb282d7815b602d32b2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37951 } } peers { permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 } } peers { permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 } } }
I20260810 18:48:15.881515 20937 tablet_replica.cc:442] T 9e867c6ca1ac44de8d4558763c678532 P 6c0f1f413ecf4bb282d7815b602d32b2: This TxnStatusTablet replica's current role is: LEADER
I20260810 18:48:15.881628 20937 tablet_replica.cc:439] T 9e867c6ca1ac44de8d4558763c678532 P 6c0f1f413ecf4bb282d7815b602d32b2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37951 } } peers { permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 } } peers { permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 } } }
I20260810 18:48:15.881716 20937 tablet_replica.cc:442] T 9e867c6ca1ac44de8d4558763c678532 P 6c0f1f413ecf4bb282d7815b602d32b2: This TxnStatusTablet replica's current role is: LEADER
I20260810 18:48:15.882071 20944 mvcc.cc:205] Tried to move back new op lower bound from 7317044002304237568 to 7317044002193854464. Current Snapshot: MvccSnapshot[applied={T|T < 7317044002304237568}]
I20260810 18:48:15.882712 20935 tablet_replica.cc:439] T 9e867c6ca1ac44de8d4558763c678532 P 3d581046937a435d9f19a8283579bc0a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37951 } } peers { permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 } } peers { permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 } } }
I20260810 18:48:15.882814 20935 tablet_replica.cc:442] T 9e867c6ca1ac44de8d4558763c678532 P 3d581046937a435d9f19a8283579bc0a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:48:15.883293 20945 mvcc.cc:205] Tried to move back new op lower bound from 7317044002304237568 to 7317044002193854464. Current Snapshot: MvccSnapshot[applied={T|T < 7317044002304237568}]
I20260810 18:48:15.883476 20936 tablet_replica.cc:439] T 9e867c6ca1ac44de8d4558763c678532 P a324512d98564e0d9babc8f4592d090c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0f1f413ecf4bb282d7815b602d32b2" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37951 } } peers { permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 } } peers { permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 } } }
I20260810 18:48:15.883620 20936 tablet_replica.cc:442] T 9e867c6ca1ac44de8d4558763c678532 P a324512d98564e0d9babc8f4592d090c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:48:15.893891 20923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:15.889401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51656 (local address 127.18.168.129:37951)
0810 18:48:15.889511 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:15.889514 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:15.889718 (+ 204us) server_negotiation.cc:407] Connection header received
0810 18:48:15.889820 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:15.889823 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:15.889875 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:15.889968 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:48:15.890264 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.891076 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.892570 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.892955 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.893060 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:15.893388 (+ 328us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:48:15.893468 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:48:15.893539 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:15.893707 (+ 168us) server_negotiation.cc:299] Negotiation successful
0810 18:48:15.893769 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 18:48:15.909085 20923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:15.905695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:44853 (local address 127.18.168.129:37951)
0810 18:48:15.905808 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:15.905814 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:15.905830 (+ 16us) server_negotiation.cc:407] Connection header received
0810 18:48:15.905876 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:15.905882 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:15.905940 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:15.906016 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:48:15.906311 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.906767 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.908011 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.908210 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.908283 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:15.908622 (+ 339us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:48:15.908727 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:48:15.908799 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:15.908922 (+ 123us) server_negotiation.cc:299] Negotiation successful
0810 18:48:15.908964 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":39}
I20260810 18:48:15.909215 20984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:15.905404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:37951 (local address 127.18.168.129:44853)
0810 18:48:15.905705 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:15.905720 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:15.905807 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:15.906043 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:15.906046 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:15.906068 (+ 22us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:48:15.906207 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:15.906213 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.906874 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.906877 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:15.907889 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:15.907900 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.908326 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.908330 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:15.908438 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:15.908488 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:48:15.908814 (+ 326us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:48:15.908820 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 18:48:15.908903 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 18:48:15.908984 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":20,"thread_start_us":101,"threads_started":1}
I20260810 18:48:15.915844 20684 raft_consensus.cc:1261] T d37bd5d2e1c943d399766c0e87f706e7 P 3d581046937a435d9f19a8283579bc0a [term 1 FOLLOWER]: Refusing update from remote peer 6c0f1f413ecf4bb282d7815b602d32b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:15.915844 20849 raft_consensus.cc:1261] T d37bd5d2e1c943d399766c0e87f706e7 P a324512d98564e0d9babc8f4592d090c [term 1 FOLLOWER]: Refusing update from remote peer 6c0f1f413ecf4bb282d7815b602d32b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:15.916226 20962 consensus_queue.cc:1035] T d37bd5d2e1c943d399766c0e87f706e7 P 6c0f1f413ecf4bb282d7815b602d32b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a324512d98564e0d9babc8f4592d090c" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:15.916385 20985 consensus_queue.cc:1035] T d37bd5d2e1c943d399766c0e87f706e7 P 6c0f1f413ecf4bb282d7815b602d32b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d581046937a435d9f19a8283579bc0a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 45727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:15.955243 20359 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39550:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:15.955448 20359 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39550:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:15.957566 20359 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 93be483b2fc741eaa27083a77fca5b6d: Sending DeleteTablet for 3 replicas of tablet d37bd5d2e1c943d399766c0e87f706e7
I20260810 18:48:15.958170 20827 tablet_service.cc:1514] Processing DeleteTablet for tablet d37bd5d2e1c943d399766c0e87f706e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:15 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:48878
I20260810 18:48:15.958237 20667 tablet_service.cc:1514] Processing DeleteTablet for tablet d37bd5d2e1c943d399766c0e87f706e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:15 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:56888
I20260810 18:48:15.958518 20956 tablet_replica.cc:330] T d37bd5d2e1c943d399766c0e87f706e7 P a324512d98564e0d9babc8f4592d090c: stopping tablet replica
I20260810 18:48:15.958535 20989 tablet_replica.cc:330] T d37bd5d2e1c943d399766c0e87f706e7 P 3d581046937a435d9f19a8283579bc0a: stopping tablet replica
I20260810 18:48:15.958654 20956 raft_consensus.cc:2229] T d37bd5d2e1c943d399766c0e87f706e7 P a324512d98564e0d9babc8f4592d090c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:15.958674 20989 raft_consensus.cc:2229] T d37bd5d2e1c943d399766c0e87f706e7 P 3d581046937a435d9f19a8283579bc0a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:15.958760 20956 raft_consensus.cc:2258] T d37bd5d2e1c943d399766c0e87f706e7 P a324512d98564e0d9babc8f4592d090c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:15.958819 20989 raft_consensus.cc:2258] T d37bd5d2e1c943d399766c0e87f706e7 P 3d581046937a435d9f19a8283579bc0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:15.959396 20989 ts_tablet_manager.cc:1905] T d37bd5d2e1c943d399766c0e87f706e7 P 3d581046937a435d9f19a8283579bc0a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:15.959594 20956 ts_tablet_manager.cc:1905] T d37bd5d2e1c943d399766c0e87f706e7 P a324512d98564e0d9babc8f4592d090c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:15.960845 20508 tablet_service.cc:1514] Processing DeleteTablet for tablet d37bd5d2e1c943d399766c0e87f706e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:15 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:51634
I20260810 18:48:15.962299 20989 ts_tablet_manager.cc:1918] T d37bd5d2e1c943d399766c0e87f706e7 P 3d581046937a435d9f19a8283579bc0a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:48:15.962306 20956 ts_tablet_manager.cc:1918] T d37bd5d2e1c943d399766c0e87f706e7 P a324512d98564e0d9babc8f4592d090c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:48:15.962390 20956 log.cc:1196] T d37bd5d2e1c943d399766c0e87f706e7 P a324512d98564e0d9babc8f4592d090c: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-2/wal/wals/d37bd5d2e1c943d399766c0e87f706e7
I20260810 18:48:15.962399 20989 log.cc:1196] T d37bd5d2e1c943d399766c0e87f706e7 P 3d581046937a435d9f19a8283579bc0a: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-1/wal/wals/d37bd5d2e1c943d399766c0e87f706e7
I20260810 18:48:15.962677 20956 ts_tablet_manager.cc:1939] T d37bd5d2e1c943d399766c0e87f706e7 P a324512d98564e0d9babc8f4592d090c: Deleting consensus metadata
I20260810 18:48:15.962713 20989 ts_tablet_manager.cc:1939] T d37bd5d2e1c943d399766c0e87f706e7 P 3d581046937a435d9f19a8283579bc0a: Deleting consensus metadata
I20260810 18:48:15.963338 20345 catalog_manager.cc:4841] TS a324512d98564e0d9babc8f4592d090c (127.18.168.131:34283): tablet d37bd5d2e1c943d399766c0e87f706e7 (table test-table [id=6356b2682c1348fbbadaf7985ac79dbb]) successfully deleted
I20260810 18:48:15.963402 20347 catalog_manager.cc:4841] TS 3d581046937a435d9f19a8283579bc0a (127.18.168.130:45727): tablet d37bd5d2e1c943d399766c0e87f706e7 (table test-table [id=6356b2682c1348fbbadaf7985ac79dbb]) successfully deleted
I20260810 18:48:15.963609 20991 tablet_replica.cc:330] T d37bd5d2e1c943d399766c0e87f706e7 P 6c0f1f413ecf4bb282d7815b602d32b2: stopping tablet replica
I20260810 18:48:15.963730 20991 raft_consensus.cc:2229] T d37bd5d2e1c943d399766c0e87f706e7 P 6c0f1f413ecf4bb282d7815b602d32b2 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:48:15.964017 20991 raft_consensus.cc:2258] T d37bd5d2e1c943d399766c0e87f706e7 P 6c0f1f413ecf4bb282d7815b602d32b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:15.964638 20991 ts_tablet_manager.cc:1905] T d37bd5d2e1c943d399766c0e87f706e7 P 6c0f1f413ecf4bb282d7815b602d32b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:15.967415 20991 ts_tablet_manager.cc:1918] T d37bd5d2e1c943d399766c0e87f706e7 P 6c0f1f413ecf4bb282d7815b602d32b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:48:15.967497 20991 log.cc:1196] T d37bd5d2e1c943d399766c0e87f706e7 P 6c0f1f413ecf4bb282d7815b602d32b2: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387687857264-19106-0/minicluster-data/ts-0/wal/wals/d37bd5d2e1c943d399766c0e87f706e7
I20260810 18:48:15.967767 20991 ts_tablet_manager.cc:1939] T d37bd5d2e1c943d399766c0e87f706e7 P 6c0f1f413ecf4bb282d7815b602d32b2: Deleting consensus metadata
I20260810 18:48:15.968513 20346 catalog_manager.cc:4841] TS 6c0f1f413ecf4bb282d7815b602d32b2 (127.18.168.129:37951): tablet d37bd5d2e1c943d399766c0e87f706e7 (table test-table [id=6356b2682c1348fbbadaf7985ac79dbb]) successfully deleted
Aug 10 18:48:15 dist-test-slave-cbsf krb5kdc[20311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387695, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.168.129@KRBTEST.COM
I20260810 18:48:15.974385 20923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:15.962921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51668 (local address 127.18.168.129:37951)
0810 18:48:15.963034 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:15.963038 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:15.963847 (+ 809us) server_negotiation.cc:407] Connection header received
0810 18:48:15.963990 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:15.963993 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:15.964035 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:15.964130 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:15.965076 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.965670 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.966845 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.967179 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.967290 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:15.970646 (+ 3356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:15.970657 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:15.970680 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:15.970703 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:15.972501 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:15.973015 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:15.973020 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:15.973021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:15.973098 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:15.973395 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:15.973398 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:15.973400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:15.973630 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:15.973848 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:15.974038 (+ 190us) server_negotiation.cc:299] Negotiation successful
0810 18:48:15.974185 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260810 18:48:15.974998 20578 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51668
I20260810 18:48:15.984648 20415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:15.976657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39558 (local address 127.18.168.190:42241)
0810 18:48:15.976827 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:15.976834 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:15.976918 (+ 84us) server_negotiation.cc:407] Connection header received
0810 18:48:15.977048 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:15.977051 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:15.977102 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:15.977223 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:15.977904 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.978686 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.979483 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:15.979817 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:15.979922 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:15.981475 (+ 1553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:15.981478 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:15.981481 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:15.981507 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:15.983036 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:15.983440 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:15.983443 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:15.983445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:15.983534 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:15.983787 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:15.983790 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:15.983792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:15.984012 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:15.984142 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:15.984368 (+ 226us) server_negotiation.cc:299] Negotiation successful
0810 18:48:15.984490 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
W20260810 18:48:15.985044 20359 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39558
I20260810 18:48:15.986388 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 20439
I20260810 18:48:15.997275 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 20596
I20260810 18:48:16.006557 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 20752
I20260810 18:48:16.013182 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 20325
2026-08-10T18:48:16Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2996 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:48:16 dist-test-slave-cbsf krb5kdc[20998](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:48:16 dist-test-slave-cbsf krb5kdc[20998](info): set up 2 sockets
Aug 10 18:48:16 dist-test-slave-cbsf krb5kdc[20998](info): commencing operation
krb5kdc: starting...
W20260810 18:48:17.133082 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:48:17 dist-test-slave-cbsf krb5kdc[20998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:48:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:48:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.168.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
I20260810 18:48:17.259116 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:36557
--webserver_interface=127.18.168.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:36481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:36557
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:17.370913 21013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:17.371253 21013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:17.371296 21013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:17.375360 21013 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:48:17.375415 21013 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 18:48:17.375432 21013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:17.375450 21013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:17.375466 21013 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:48:17.375481 21013 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:48:17.380649 21013 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:36481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:36557
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:36557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:17.381695 21013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:17.382647 21013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:17.391356 21023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:17.391360 21021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:17.391492 21013 server_base.cc:1034] running on GCE node
W20260810 18:48:17.391367 21020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:17.392136 21013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:17.393285 21013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:17.394505 21013 hybrid_clock.cc:648] HybridClock initialized: now 1786387697394480 us; error 37 us; skew 500 ppm
Aug 10 18:48:17 dist-test-slave-cbsf krb5kdc[20998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387697, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:17.397290 21013 init.cc:377] Logged in from keytab as kudu/127.18.168.190@KRBTEST.COM (short username kudu)
I20260810 18:48:17.398100 21013 webserver.cc:460] Webserver started at http://127.18.168.190:36651/ using document root <none> and password file <none>
I20260810 18:48:17.398602 21013 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:17.398660 21013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:17.398819 21013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:17.400900 21013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "fbe4cd70c6a54bdb8500cd1b1369f5be"
format_stamp: "Formatted at 2026-08-10 18:48:17 on dist-test-slave-cbsf"
server_key: "c88919f88efea87f9afcefc282ef2826"
server_key_iv: "4ab3e026c9f51dddb51bc361ac289f8b"
server_key_version: "encryptionkey@0"
I20260810 18:48:17.401432 21013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "fbe4cd70c6a54bdb8500cd1b1369f5be"
format_stamp: "Formatted at 2026-08-10 18:48:17 on dist-test-slave-cbsf"
server_key: "c88919f88efea87f9afcefc282ef2826"
server_key_iv: "4ab3e026c9f51dddb51bc361ac289f8b"
server_key_version: "encryptionkey@0"
I20260810 18:48:17.405277 21013 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 18:48:17.407683 21031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:17.408980 21013 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:48:17.409124 21013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "fbe4cd70c6a54bdb8500cd1b1369f5be"
format_stamp: "Formatted at 2026-08-10 18:48:17 on dist-test-slave-cbsf"
server_key: "c88919f88efea87f9afcefc282ef2826"
server_key_iv: "4ab3e026c9f51dddb51bc361ac289f8b"
server_key_version: "encryptionkey@0"
I20260810 18:48:17.409235 21013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:17.438702 21013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:17.441730 21013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:17.441938 21013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:17.450098 21013 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:36557
I20260810 18:48:17.450122 21100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:36557 every 8 connection(s)
I20260810 18:48:17.451045 21013 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:48:17.453945 21101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:17.455261 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 21013
I20260810 18:48:17.455353 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:48:17.455637 19106 external_mini_cluster.cc:1428] Setting key e2a333d2a4d48255b0d6c5e8a8c5020c
I20260810 18:48:17.460868 21101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbe4cd70c6a54bdb8500cd1b1369f5be: Bootstrap starting.
I20260810 18:48:17.463608 21101 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fbe4cd70c6a54bdb8500cd1b1369f5be: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:17.464371 21101 log.cc:824] T 00000000000000000000000000000000 P fbe4cd70c6a54bdb8500cd1b1369f5be: Log is configured to *not* fsync() on all Append() calls
Aug 10 18:48:17 dist-test-slave-cbsf krb5kdc[20998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:17.466717 21101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbe4cd70c6a54bdb8500cd1b1369f5be: No bootstrap required, opened a new log
I20260810 18:48:17.471122 21101 raft_consensus.cc:348] T 00000000000000000000000000000000 P fbe4cd70c6a54bdb8500cd1b1369f5be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe4cd70c6a54bdb8500cd1b1369f5be" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 36557 } }
I20260810 18:48:17.471434 21101 raft_consensus.cc:374] T 00000000000000000000000000000000 P fbe4cd70c6a54bdb8500cd1b1369f5be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:17.471493 21101 raft_consensus.cc:729] T 00000000000000000000000000000000 P fbe4cd70c6a54bdb8500cd1b1369f5be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbe4cd70c6a54bdb8500cd1b1369f5be, State: Initialized, Role: FOLLOWER
I20260810 18:48:17.471994 21106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:17.457134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50030 (local address 127.18.168.190:36557)
0810 18:48:17.457599 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:17.457620 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:17.457674 (+ 54us) server_negotiation.cc:407] Connection header received
0810 18:48:17.458491 (+ 817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:17.458516 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:17.458952 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:17.459315 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:17.459990 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:17.461420 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:17.462265 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:17.462962 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:17.463121 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:17.465892 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:17.465898 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:17.465915 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:17.465947 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:17.468754 (+ 2807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:17.469173 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:17.469180 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:17.469186 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:17.469325 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:17.469592 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:17.469595 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:17.469597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:17.470124 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:17.470338 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:17.470528 (+ 190us) server_negotiation.cc:299] Negotiation successful
0810 18:48:17.470844 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":127,"threads_started":1}
I20260810 18:48:17.471961 21101 consensus_queue.cc:260] T 00000000000000000000000000000000 P fbe4cd70c6a54bdb8500cd1b1369f5be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe4cd70c6a54bdb8500cd1b1369f5be" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 36557 } }
I20260810 18:48:17.472121 21101 raft_consensus.cc:388] T 00000000000000000000000000000000 P fbe4cd70c6a54bdb8500cd1b1369f5be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:48:17.472203 21101 raft_consensus.cc:482] T 00000000000000000000000000000000 P fbe4cd70c6a54bdb8500cd1b1369f5be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:48:17.472291 21101 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fbe4cd70c6a54bdb8500cd1b1369f5be [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:17.473323 21101 raft_consensus.cc:504] T 00000000000000000000000000000000 P fbe4cd70c6a54bdb8500cd1b1369f5be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe4cd70c6a54bdb8500cd1b1369f5be" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 36557 } }
I20260810 18:48:17.473670 21101 leader_election.cc:302] T 00000000000000000000000000000000 P fbe4cd70c6a54bdb8500cd1b1369f5be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fbe4cd70c6a54bdb8500cd1b1369f5be; no voters:
I20260810 18:48:17.473991 21101 leader_election.cc:288] T 00000000000000000000000000000000 P fbe4cd70c6a54bdb8500cd1b1369f5be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:48:17.474112 21108 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fbe4cd70c6a54bdb8500cd1b1369f5be [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:17.474365 21108 raft_consensus.cc:686] T 00000000000000000000000000000000 P fbe4cd70c6a54bdb8500cd1b1369f5be [term 1 LEADER]: Becoming Leader. State: Replica: fbe4cd70c6a54bdb8500cd1b1369f5be, State: Running, Role: LEADER
I20260810 18:48:17.474668 21108 consensus_queue.cc:237] T 00000000000000000000000000000000 P fbe4cd70c6a54bdb8500cd1b1369f5be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe4cd70c6a54bdb8500cd1b1369f5be" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 36557 } }
I20260810 18:48:17.475251 21101 sys_catalog.cc:564] T 00000000000000000000000000000000 P fbe4cd70c6a54bdb8500cd1b1369f5be [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:48:17.476468 21110 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbe4cd70c6a54bdb8500cd1b1369f5be [sys.catalog]: SysCatalogTable state changed. Reason: New leader fbe4cd70c6a54bdb8500cd1b1369f5be. Latest consensus state: current_term: 1 leader_uuid: "fbe4cd70c6a54bdb8500cd1b1369f5be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe4cd70c6a54bdb8500cd1b1369f5be" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 36557 } } }
I20260810 18:48:17.476622 21110 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbe4cd70c6a54bdb8500cd1b1369f5be [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:17.476480 21109 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbe4cd70c6a54bdb8500cd1b1369f5be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fbe4cd70c6a54bdb8500cd1b1369f5be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe4cd70c6a54bdb8500cd1b1369f5be" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 36557 } } }
I20260810 18:48:17.476882 21109 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbe4cd70c6a54bdb8500cd1b1369f5be [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:17.477064 21118 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:48:17.480180 21118 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:48:17.486507 21118 catalog_manager.cc:1347] Generated new cluster ID: 98cc247968004a168d56eb2224cc4cad
I20260810 18:48:17.486585 21118 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:48:17.501510 21118 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:48:17.502509 21118 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:48:17.518373 21118 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fbe4cd70c6a54bdb8500cd1b1369f5be: Generated new TSK 0
I20260810 18:48:17.519138 21118 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.168.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab.
Entry for principal kudu/127.18.168.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab.
I20260810 18:48:17.557649 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.129:0
--local_ip_for_outbound_sockets=127.18.168.129
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:36557
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:36481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:17.679428 21130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:17.679690 21130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:17.679735 21130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:17.683682 21130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:17.683735 21130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:17.683772 21130 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 18:48:17.683786 21130 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 18:48:17.683822 21130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.129
I20260810 18:48:17.688898 21130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:36481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.168.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.168.190:36557
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.129
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:17.689834 21130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:17.690743 21130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:17.700875 21137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:17.700922 21139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:17.700884 21136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:17.701269 21130 server_base.cc:1034] running on GCE node
I20260810 18:48:17.701701 21130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:17.702389 21130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:17.703593 21130 hybrid_clock.cc:648] HybridClock initialized: now 1786387697703575 us; error 27 us; skew 500 ppm
Aug 10 18:48:17 dist-test-slave-cbsf krb5kdc[20998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387697, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:17.706728 21130 init.cc:377] Logged in from keytab as kudu/127.18.168.129@KRBTEST.COM (short username kudu)
I20260810 18:48:17.707635 21130 webserver.cc:460] Webserver started at http://127.18.168.129:35879/ using document root <none> and password file <none>
I20260810 18:48:17.708238 21130 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:17.708341 21130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:17.708591 21130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:17.710789 21130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-0/data/instance:
uuid: "ec02cf09b8914f118a120895e3c0f3cf"
format_stamp: "Formatted at 2026-08-10 18:48:17 on dist-test-slave-cbsf"
server_key: "f0b760b39e9d69c3080cd5fe220400d3"
server_key_iv: "77b3a63a8b231ded020f8df6da329839"
server_key_version: "encryptionkey@0"
I20260810 18:48:17.711410 21130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance:
uuid: "ec02cf09b8914f118a120895e3c0f3cf"
format_stamp: "Formatted at 2026-08-10 18:48:17 on dist-test-slave-cbsf"
server_key: "f0b760b39e9d69c3080cd5fe220400d3"
server_key_iv: "77b3a63a8b231ded020f8df6da329839"
server_key_version: "encryptionkey@0"
I20260810 18:48:17.715471 21130 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:48:17.718071 21148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:17.719370 21130 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:48:17.719515 21130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-0/wal
uuid: "ec02cf09b8914f118a120895e3c0f3cf"
format_stamp: "Formatted at 2026-08-10 18:48:17 on dist-test-slave-cbsf"
server_key: "f0b760b39e9d69c3080cd5fe220400d3"
server_key_iv: "77b3a63a8b231ded020f8df6da329839"
server_key_version: "encryptionkey@0"
I20260810 18:48:17.719651 21130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:17.742303 21130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:17.746054 21130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:17.746296 21130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:17.747869 21130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:17.747948 21130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:17.748042 21130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:17.748104 21130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 18:48:17 dist-test-slave-cbsf krb5kdc[20998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387697, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:17.763549 21106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:17.748795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:41709 (local address 127.18.168.190:36557)
0810 18:48:17.748942 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:17.748946 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:17.750077 (+ 1131us) server_negotiation.cc:407] Connection header received
0810 18:48:17.751141 (+ 1064us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:17.751146 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:17.751216 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:17.751323 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:17.752619 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:17.753640 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:17.754913 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:17.755225 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:17.755352 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:17.759065 (+ 3713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:17.759072 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:17.759077 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:17.759140 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:17.761139 (+ 1999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:17.761674 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:17.761680 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:17.761683 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:17.761795 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:17.762231 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:17.762236 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:17.762240 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:17.762521 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:17.762618 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:17.763158 (+ 540us) server_negotiation.cc:299] Negotiation successful
0810 18:48:17.763313 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 18:48:17.763567 21130 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.129:41069
I20260810 18:48:17.763789 21274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.129:41069 every 8 connection(s)
I20260810 18:48:17.764443 21157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:17.749415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:36557 (local address 127.18.168.129:41709)
0810 18:48:17.749925 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:17.749968 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:17.750879 (+ 911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:17.751603 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:17.751612 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:17.751995 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:17.752482 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:17.752495 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:17.753803 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:17.753807 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:17.754780 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:17.754791 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:17.755406 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:17.755410 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:17.755580 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:17.756349 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:17.756368 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:17.758864 (+ 2496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:17.761267 (+ 2403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:17.761274 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:17.761289 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:17.761552 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:17.761943 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:17.761947 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:17.761951 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:17.762103 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:17.762634 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:17.762639 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:17.762883 (+ 244us) client_negotiation.cc:769] Sending connection context
0810 18:48:17.763059 (+ 176us) client_negotiation.cc:240] Negotiation successful
0810 18:48:17.763455 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":152,"threads_started":1}
I20260810 18:48:17.764467 21130 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
I20260810 18:48:17.765350 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 21130
I20260810 18:48:17.765497 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance
I20260810 18:48:17.765832 19106 external_mini_cluster.cc:1428] Setting key da9d4a99b4b743e92226ffd4082e2af9
I20260810 18:48:17.769549 21277 heartbeater.cc:347] Connected to a master server at 127.18.168.190:36557
I20260810 18:48:17.769729 21277 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:17.770316 21277 heartbeater.cc:511] Master 127.18.168.190:36557 requested a full tablet report, sending...
I20260810 18:48:17.772367 21051 ts_manager.cc:194] Registered new tserver with Master: ec02cf09b8914f118a120895e3c0f3cf (127.18.168.129:41069)
I20260810 18:48:17.774154 21051 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.129@KRBTEST.COM'} at 127.18.168.129:41709
WARNING: no policy specified for kudu/127.18.168.130@KRBTEST.COM; defaulting to no policy
I20260810 18:48:17.779183 21106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:17.768790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:38961 (local address 127.18.168.190:36557)
0810 18:48:17.768913 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:17.768918 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:17.770289 (+ 1371us) server_negotiation.cc:407] Connection header received
0810 18:48:17.770401 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:17.770406 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:17.770462 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:17.770551 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:17.771280 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:17.772128 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:17.773435 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:17.773823 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:17.773896 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:17.775290 (+ 1394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:17.775294 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:17.775297 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:17.775323 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:17.777426 (+ 2103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:17.777919 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:17.777923 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:17.777924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:17.778015 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:17.778311 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:17.778315 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:17.778317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:17.778531 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:17.778647 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:17.778917 (+ 270us) server_negotiation.cc:299] Negotiation successful
0810 18:48:17.779032 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 18:48:17.779201 21286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:17.768770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:36557 (local address 127.18.168.129:38961)
0810 18:48:17.770183 (+ 1413us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:17.770199 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:17.770307 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:17.770595 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:17.770598 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:17.770830 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:17.770969 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:17.770984 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:17.772310 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:17.772314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:17.773274 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:17.773287 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:17.774061 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:17.774064 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:17.774163 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:17.774704 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:17.774720 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:17.775134 (+ 414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:17.777589 (+ 2455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:17.777593 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:17.777595 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:17.777812 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:17.778111 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:17.778115 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:17.778117 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:17.778209 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:17.778637 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:17.778639 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:17.778738 (+ 99us) client_negotiation.cc:769] Sending connection context
0810 18:48:17.778851 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 18:48:17.779001 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1313,"thread_start_us":96,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab.
Entry for principal kudu/127.18.168.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab.
I20260810 18:48:17.797392 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.130:0
--local_ip_for_outbound_sockets=127.18.168.130
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:36557
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:36481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:17.918306 21289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:17.918566 21289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:17.918660 21289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:17.923218 21289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:17.923269 21289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:17.923337 21289 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 18:48:17.923370 21289 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 18:48:17.923558 21289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.130
I20260810 18:48:17.929220 21289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:36481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.168.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.168.190:36557
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21289
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.130
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:17.930318 21289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:17.931384 21289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:17.941149 21295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:17.941145 21294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:17.941331 21297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:17.941190 21289 server_base.cc:1034] running on GCE node
I20260810 18:48:17.941785 21289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:17.942422 21289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:17.943620 21289 hybrid_clock.cc:648] HybridClock initialized: now 1786387697943603 us; error 27 us; skew 500 ppm
Aug 10 18:48:17 dist-test-slave-cbsf krb5kdc[20998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387697, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:17.946460 21289 init.cc:377] Logged in from keytab as kudu/127.18.168.130@KRBTEST.COM (short username kudu)
I20260810 18:48:17.947365 21289 webserver.cc:460] Webserver started at http://127.18.168.130:39561/ using document root <none> and password file <none>
I20260810 18:48:17.947949 21289 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:17.948040 21289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:17.948274 21289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:17.950416 21289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-1/data/instance:
uuid: "1c06a8266d164fdc98b6891041be38d0"
format_stamp: "Formatted at 2026-08-10 18:48:17 on dist-test-slave-cbsf"
server_key: "4a8b1889d08af33385ea300c40816188"
server_key_iv: "144e06efeba28e29b1d462aef334de6e"
server_key_version: "encryptionkey@0"
I20260810 18:48:17.951000 21289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance:
uuid: "1c06a8266d164fdc98b6891041be38d0"
format_stamp: "Formatted at 2026-08-10 18:48:17 on dist-test-slave-cbsf"
server_key: "4a8b1889d08af33385ea300c40816188"
server_key_iv: "144e06efeba28e29b1d462aef334de6e"
server_key_version: "encryptionkey@0"
I20260810 18:48:17.954849 21289 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260810 18:48:17.957276 21304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:17.958498 21289 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:48:17.958663 21289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-1/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-1/wal
uuid: "1c06a8266d164fdc98b6891041be38d0"
format_stamp: "Formatted at 2026-08-10 18:48:17 on dist-test-slave-cbsf"
server_key: "4a8b1889d08af33385ea300c40816188"
server_key_iv: "144e06efeba28e29b1d462aef334de6e"
server_key_version: "encryptionkey@0"
I20260810 18:48:17.958776 21289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:17.976137 21289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:17.979375 21289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:17.979588 21289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:17.981155 21289 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:17.981241 21289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:17.981316 21289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:17.981351 21289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 18:48:17 dist-test-slave-cbsf krb5kdc[20998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387697, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:17.992365 21289 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.130:35203
I20260810 18:48:17.993228 21435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.130:35203 every 8 connection(s)
I20260810 18:48:17.993398 21289 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
I20260810 18:48:17.994716 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 21289
I20260810 18:48:17.994828 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance
I20260810 18:48:17.995188 19106 external_mini_cluster.cc:1428] Setting key 60a132a3faa0d919afc01a266aab4ba2
I20260810 18:48:17.995424 21106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:17.981937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:32891 (local address 127.18.168.190:36557)
0810 18:48:17.982102 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:17.982107 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:17.983005 (+ 898us) server_negotiation.cc:407] Connection header received
0810 18:48:17.984221 (+ 1216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:17.984228 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:17.984300 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:17.984406 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:17.986051 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:17.986884 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:17.987988 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:17.988323 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:17.988449 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:17.991269 (+ 2820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:17.991274 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:17.991277 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:17.991312 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:17.992953 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:17.993570 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:17.993573 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:17.993575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:17.993672 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:17.994120 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:17.994124 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:17.994126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:17.994377 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:17.994505 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:17.995134 (+ 629us) server_negotiation.cc:299] Negotiation successful
0810 18:48:17.995273 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 18:48:17.996382 21314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:17.982379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:36557 (local address 127.18.168.130:32891)
0810 18:48:17.982835 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:17.982888 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:17.983966 (+ 1078us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:17.984672 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:17.984681 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:17.985168 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:17.985881 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:17.985920 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:17.987005 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:17.987008 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:17.987857 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:17.987866 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:17.988504 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:17.988508 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:17.988748 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:17.989372 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:17.989389 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:17.991099 (+ 1710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:17.993069 (+ 1970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:17.993075 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:17.993092 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:17.993462 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:17.993780 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:17.993784 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:17.993786 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:17.994001 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:17.994524 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:17.994532 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:17.994845 (+ 313us) client_negotiation.cc:769] Sending connection context
0810 18:48:17.995097 (+ 252us) client_negotiation.cc:240] Negotiation successful
0810 18:48:17.995425 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":136,"threads_started":1}
I20260810 18:48:17.997689 21436 heartbeater.cc:347] Connected to a master server at 127.18.168.190:36557
I20260810 18:48:17.997850 21436 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:17.998420 21436 heartbeater.cc:511] Master 127.18.168.190:36557 requested a full tablet report, sending...
I20260810 18:48:17.999473 21051 ts_manager.cc:194] Registered new tserver with Master: 1c06a8266d164fdc98b6891041be38d0 (127.18.168.130:35203)
I20260810 18:48:18.000155 21051 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.130@KRBTEST.COM'} at 127.18.168.130:32891
WARNING: no policy specified for kudu/127.18.168.131@KRBTEST.COM; defaulting to no policy
I20260810 18:48:18.011004 21106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:18.000307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:53897 (local address 127.18.168.190:36557)
0810 18:48:18.000449 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:18.000456 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:18.000471 (+ 15us) server_negotiation.cc:407] Connection header received
0810 18:48:18.000513 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:18.000518 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:18.000567 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:18.000657 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:18.004449 (+ 3792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.005395 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.006216 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.006606 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.006717 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:18.007835 (+ 1118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:18.007839 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:18.007841 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:18.007868 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:18.009372 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:18.009887 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:18.009889 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:18.009890 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:18.009965 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:18.010209 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:18.010211 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:18.010212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:18.010414 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:18.010517 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:18.010781 (+ 264us) server_negotiation.cc:299] Negotiation successful
0810 18:48:18.010876 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 18:48:18.011016 21445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:17.999936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:36557 (local address 127.18.168.130:53897)
0810 18:48:18.000302 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:18.000318 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:18.000409 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:18.000713 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:18.000717 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:18.004199 (+ 3482us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:18.004316 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:18.004323 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.005524 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.005527 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:18.006081 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:18.006089 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.006730 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.006733 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:18.006802 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:18.007355 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:18.007366 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:18.007681 (+ 315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:18.009565 (+ 1884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:18.009572 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:18.009577 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:18.009798 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:18.010051 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:18.010053 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:18.010054 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:18.010130 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:18.010497 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:18.010499 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:18.010641 (+ 142us) client_negotiation.cc:769] Sending connection context
0810 18:48:18.010728 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 18:48:18.010836 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":103,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab.
Entry for principal kudu/127.18.168.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab.
I20260810 18:48:18.027841 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.131:0
--local_ip_for_outbound_sockets=127.18.168.131
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:36557
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:36481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:18.148223 21447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:18.148530 21447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:18.148631 21447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:18.152665 21447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:18.152774 21447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:18.152846 21447 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 18:48:18.152889 21447 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 18:48:18.152942 21447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.131
I20260810 18:48:18.158177 21447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:36481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.168.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.168.190:36557
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.131
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:18.159255 21447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:18.160401 21447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:18.170553 21453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:18.170658 21447 server_base.cc:1034] running on GCE node
W20260810 18:48:18.170562 21456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:18.170588 21454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:18.171427 21447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:18.172071 21447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:18.173260 21447 hybrid_clock.cc:648] HybridClock initialized: now 1786387698173239 us; error 30 us; skew 500 ppm
Aug 10 18:48:18 dist-test-slave-cbsf krb5kdc[20998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387698, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:18.176297 21447 init.cc:377] Logged in from keytab as kudu/127.18.168.131@KRBTEST.COM (short username kudu)
I20260810 18:48:18.177240 21447 webserver.cc:460] Webserver started at http://127.18.168.131:36467/ using document root <none> and password file <none>
I20260810 18:48:18.177805 21447 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:18.177866 21447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:18.178107 21447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:18.180079 21447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-2/data/instance:
uuid: "6b51c58c505c44c7943c02c022ddfa18"
format_stamp: "Formatted at 2026-08-10 18:48:18 on dist-test-slave-cbsf"
server_key: "9ab000f0796389d921451f3e27db3675"
server_key_iv: "44b29213a42dfe97092419d3e27b72f2"
server_key_version: "encryptionkey@0"
I20260810 18:48:18.180639 21447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance:
uuid: "6b51c58c505c44c7943c02c022ddfa18"
format_stamp: "Formatted at 2026-08-10 18:48:18 on dist-test-slave-cbsf"
server_key: "9ab000f0796389d921451f3e27db3675"
server_key_iv: "44b29213a42dfe97092419d3e27b72f2"
server_key_version: "encryptionkey@0"
I20260810 18:48:18.184151 21447 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:48:18.186681 21463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:18.187810 21447 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:48:18.187947 21447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-2/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-2/wal
uuid: "6b51c58c505c44c7943c02c022ddfa18"
format_stamp: "Formatted at 2026-08-10 18:48:18 on dist-test-slave-cbsf"
server_key: "9ab000f0796389d921451f3e27db3675"
server_key_iv: "44b29213a42dfe97092419d3e27b72f2"
server_key_version: "encryptionkey@0"
I20260810 18:48:18.188036 21447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:18.215632 21447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:18.219260 21447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:18.219480 21447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:18.221128 21447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:18.221202 21447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:18.221271 21447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:18.221328 21447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 18:48:18 dist-test-slave-cbsf krb5kdc[20998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387698, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:18.232934 21447 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.131:46491
I20260810 18:48:18.233007 21596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.131:46491 every 8 connection(s)
I20260810 18:48:18.234016 21447 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
I20260810 18:48:18.235579 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 21447
I20260810 18:48:18.235697 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance
I20260810 18:48:18.235702 21106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:18.221839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:38747 (local address 127.18.168.190:36557)
0810 18:48:18.222034 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:18.222041 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:18.223512 (+ 1471us) server_negotiation.cc:407] Connection header received
0810 18:48:18.225076 (+ 1564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:18.225079 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:18.225152 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:18.225260 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:18.226574 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.227230 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.228591 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.228916 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.229037 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:18.231616 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:18.231620 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:18.231623 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:18.231652 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:18.233318 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:18.233908 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:18.233911 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:18.233912 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:18.233987 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:18.234366 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:18.234369 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:18.234370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:18.234639 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:18.234750 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:18.235353 (+ 603us) server_negotiation.cc:299] Negotiation successful
0810 18:48:18.235505 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 18:48:18.236039 19106 external_mini_cluster.cc:1428] Setting key b09a2ada5349a3f30b6f35140df11c5f
I20260810 18:48:18.236418 21474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:18.222730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:36557 (local address 127.18.168.131:38747)
0810 18:48:18.223300 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:18.223360 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:18.224820 (+ 1460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:18.225501 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:18.225508 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:18.225897 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:18.226398 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:18.226412 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.227371 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.227375 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:18.228447 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:18.228475 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.229088 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.229091 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:18.229279 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:18.229899 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:18.229915 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:18.231442 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:18.233432 (+ 1990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:18.233438 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:18.233451 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:18.233799 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:18.234080 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:18.234084 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:18.234087 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:18.234249 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:18.234804 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:18.234810 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:18.235068 (+ 258us) client_negotiation.cc:769] Sending connection context
0810 18:48:18.235225 (+ 157us) client_negotiation.cc:240] Negotiation successful
0810 18:48:18.235471 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":167,"threads_started":1}
I20260810 18:48:18.237830 21597 heartbeater.cc:347] Connected to a master server at 127.18.168.190:36557
I20260810 18:48:18.237982 21597 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:18.238623 21597 heartbeater.cc:511] Master 127.18.168.190:36557 requested a full tablet report, sending...
I20260810 18:48:18.239634 21050 ts_manager.cc:194] Registered new tserver with Master: 6b51c58c505c44c7943c02c022ddfa18 (127.18.168.131:46491)
I20260810 18:48:18.240381 21050 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.131@KRBTEST.COM'} at 127.18.168.131:38747
I20260810 18:48:18.247319 21106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:18.240002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:37029 (local address 127.18.168.190:36557)
0810 18:48:18.240123 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:18.240127 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:18.240863 (+ 736us) server_negotiation.cc:407] Connection header received
0810 18:48:18.240946 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:18.240951 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:18.240997 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:18.241101 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:18.241519 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.242019 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.242812 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.243039 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.243147 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:18.244230 (+ 1083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:18.244234 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:18.244236 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:18.244259 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:18.245737 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:18.246176 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:18.246179 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:18.246181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:18.246258 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:18.246540 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:18.246544 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:18.246545 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:18.246709 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:18.246812 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:18.247070 (+ 258us) server_negotiation.cc:299] Negotiation successful
0810 18:48:18.247170 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 18:48:18.247336 21604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:18.239940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:36557 (local address 127.18.168.131:37029)
0810 18:48:18.240771 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:18.240793 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:18.240871 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:18.241117 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:18.241120 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:18.241316 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:18.241404 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:18.241409 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.242151 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.242155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:18.242697 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:18.242705 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.243152 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.243155 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:18.243244 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:18.243742 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:18.243755 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:18.244092 (+ 337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:18.245853 (+ 1761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:18.245858 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:18.245860 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:18.246075 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:18.246351 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:18.246355 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:18.246356 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:18.246442 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:18.246816 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:18.246820 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:18.246910 (+ 90us) client_negotiation.cc:769] Sending connection context
0810 18:48:18.247014 (+ 104us) client_negotiation.cc:240] Negotiation successful
0810 18:48:18.247160 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"thread_start_us":84,"threads_started":1}
I20260810 18:48:18.250119 19106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 18:48:18 dist-test-slave-cbsf krb5kdc[20998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 18:48:18 dist-test-slave-cbsf krb5kdc[20998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:18.276484 21106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:18.268242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50034 (local address 127.18.168.190:36557)
0810 18:48:18.268377 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:18.268380 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:18.268391 (+ 11us) server_negotiation.cc:407] Connection header received
0810 18:48:18.268428 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:18.268430 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:18.268465 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:18.268595 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:18.269349 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.269956 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.270795 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.271012 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.271119 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:18.273502 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:18.273505 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:18.273507 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:18.273536 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:18.274822 (+ 1286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:18.275284 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:18.275288 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:18.275290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:18.275383 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:18.275649 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:18.275652 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:18.275653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:18.275822 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:18.275924 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:18.276200 (+ 276us) server_negotiation.cc:299] Negotiation successful
0810 18:48:18.276351 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 18:48:18.279184 21050 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50034:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:48:18.282524 21050 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:48:18.296976 21618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:18.290846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38568 (local address 127.18.168.129:41069)
0810 18:48:18.292148 (+ 1302us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:18.292153 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:18.292171 (+ 18us) server_negotiation.cc:407] Connection header received
0810 18:48:18.292253 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:18.292258 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:18.292455 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:18.292736 (+ 281us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:18.293227 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.293882 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.295706 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.296248 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.296387 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:18.296518 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:18.296725 (+ 207us) server_negotiation.cc:299] Negotiation successful
0810 18:48:18.296825 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1205,"thread_start_us":91,"threads_started":1}
I20260810 18:48:18.297005 21616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:18.290726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:41069 (local address 127.0.0.1:38568)
0810 18:48:18.291338 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:18.291373 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:18.291518 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:18.292610 (+ 1092us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:18.292613 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:18.292652 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:18.293098 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:18.293105 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.293999 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.294003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:18.295532 (+ 1529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:18.295544 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.296375 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.296379 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:18.296486 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:18.296536 (+ 50us) client_negotiation.cc:769] Sending connection context
0810 18:48:18.296670 (+ 134us) client_negotiation.cc:240] Negotiation successful
0810 18:48:18.296825 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"mutex_wait_us":395,"spinlock_wait_cycles":10880,"thread_start_us":62,"threads_started":1}
I20260810 18:48:18.297052 21617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:18.290816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:35203 (local address 127.0.0.1:48192)
0810 18:48:18.291836 (+ 1020us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:18.291850 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:18.291933 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:18.292607 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:18.292612 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:18.292644 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:18.292815 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:18.292824 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.294191 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.294195 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:18.295537 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:18.295544 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.296574 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.296578 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:18.296676 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:18.296728 (+ 52us) client_negotiation.cc:769] Sending connection context
0810 18:48:18.296780 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 18:48:18.296861 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":952,"mutex_wait_us":74,"spinlock_wait_cycles":2816,"thread_start_us":88,"threads_started":1}
I20260810 18:48:18.297497 21619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:18.291448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.131:46491 (local address 127.0.0.1:41532)
0810 18:48:18.292339 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:18.292349 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:18.292461 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:18.292925 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:18.292927 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:18.292944 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:18.293124 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:18.293134 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.294470 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.294473 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:18.295384 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:18.295395 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.296729 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.296732 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:18.296832 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:18.296850 (+ 18us) client_negotiation.cc:769] Sending connection context
0810 18:48:18.296913 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 18:48:18.297355 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":841,"mutex_wait_us":41,"thread_start_us":99,"threads_started":1}
I20260810 18:48:18.297700 21621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:18.291744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41532 (local address 127.18.168.131:46491)
0810 18:48:18.292195 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:18.292209 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:18.292440 (+ 231us) server_negotiation.cc:407] Connection header received
0810 18:48:18.292571 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:18.292576 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:18.292776 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:18.292932 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:18.293269 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.294339 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.295511 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.296347 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.296491 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:18.296630 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:18.297049 (+ 419us) server_negotiation.cc:299] Negotiation successful
0810 18:48:18.297154 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":103,"threads_started":1}
I20260810 18:48:18.297966 21620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:18.291683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48192 (local address 127.18.168.130:35203)
0810 18:48:18.292104 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:18.292117 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:18.292142 (+ 25us) server_negotiation.cc:407] Connection header received
0810 18:48:18.292292 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:18.292296 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:18.292454 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:18.292618 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:18.292953 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.294081 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.295679 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.296444 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.297389 (+ 945us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:18.297512 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:18.297701 (+ 189us) server_negotiation.cc:299] Negotiation successful
0810 18:48:18.297830 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":110,"threads_started":1}
I20260810 18:48:18.300558 21361 tablet_service.cc:1467] Processing CreateTablet for tablet 0d911577ce40499f93ab70176722159f (DEFAULT_TABLE table=test-table [id=f925a305f263443fa86d3b9223916a52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:18.300576 21520 tablet_service.cc:1467] Processing CreateTablet for tablet 0d911577ce40499f93ab70176722159f (DEFAULT_TABLE table=test-table [id=f925a305f263443fa86d3b9223916a52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:18.300575 21201 tablet_service.cc:1467] Processing CreateTablet for tablet 0d911577ce40499f93ab70176722159f (DEFAULT_TABLE table=test-table [id=f925a305f263443fa86d3b9223916a52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:18.301679 21520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d911577ce40499f93ab70176722159f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:18.301676 21201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d911577ce40499f93ab70176722159f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:18.301705 21361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d911577ce40499f93ab70176722159f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:18.308604 21622 tablet_bootstrap.cc:492] T 0d911577ce40499f93ab70176722159f P 1c06a8266d164fdc98b6891041be38d0: Bootstrap starting.
I20260810 18:48:18.309631 21624 tablet_bootstrap.cc:492] T 0d911577ce40499f93ab70176722159f P ec02cf09b8914f118a120895e3c0f3cf: Bootstrap starting.
I20260810 18:48:18.310014 21625 tablet_bootstrap.cc:492] T 0d911577ce40499f93ab70176722159f P 6b51c58c505c44c7943c02c022ddfa18: Bootstrap starting.
I20260810 18:48:18.311364 21622 tablet_bootstrap.cc:654] T 0d911577ce40499f93ab70176722159f P 1c06a8266d164fdc98b6891041be38d0: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:18.311661 21625 tablet_bootstrap.cc:654] T 0d911577ce40499f93ab70176722159f P 6b51c58c505c44c7943c02c022ddfa18: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:18.312337 21622 log.cc:824] T 0d911577ce40499f93ab70176722159f P 1c06a8266d164fdc98b6891041be38d0: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:18.312398 21625 log.cc:824] T 0d911577ce40499f93ab70176722159f P 6b51c58c505c44c7943c02c022ddfa18: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:18.313730 21624 tablet_bootstrap.cc:654] T 0d911577ce40499f93ab70176722159f P ec02cf09b8914f118a120895e3c0f3cf: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:18.314241 21625 tablet_bootstrap.cc:492] T 0d911577ce40499f93ab70176722159f P 6b51c58c505c44c7943c02c022ddfa18: No bootstrap required, opened a new log
I20260810 18:48:18.314455 21625 ts_tablet_manager.cc:1397] T 0d911577ce40499f93ab70176722159f P 6b51c58c505c44c7943c02c022ddfa18: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20260810 18:48:18.314507 21622 tablet_bootstrap.cc:492] T 0d911577ce40499f93ab70176722159f P 1c06a8266d164fdc98b6891041be38d0: No bootstrap required, opened a new log
I20260810 18:48:18.314589 21624 log.cc:824] T 0d911577ce40499f93ab70176722159f P ec02cf09b8914f118a120895e3c0f3cf: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:18.314704 21622 ts_tablet_manager.cc:1397] T 0d911577ce40499f93ab70176722159f P 1c06a8266d164fdc98b6891041be38d0: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260810 18:48:18.316542 21624 tablet_bootstrap.cc:492] T 0d911577ce40499f93ab70176722159f P ec02cf09b8914f118a120895e3c0f3cf: No bootstrap required, opened a new log
I20260810 18:48:18.316751 21624 ts_tablet_manager.cc:1397] T 0d911577ce40499f93ab70176722159f P ec02cf09b8914f118a120895e3c0f3cf: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260810 18:48:18.317838 21625 raft_consensus.cc:348] T 0d911577ce40499f93ab70176722159f P 6b51c58c505c44c7943c02c022ddfa18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:18.318074 21625 raft_consensus.cc:374] T 0d911577ce40499f93ab70176722159f P 6b51c58c505c44c7943c02c022ddfa18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:18.318127 21625 raft_consensus.cc:729] T 0d911577ce40499f93ab70176722159f P 6b51c58c505c44c7943c02c022ddfa18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b51c58c505c44c7943c02c022ddfa18, State: Initialized, Role: FOLLOWER
I20260810 18:48:18.318576 21625 consensus_queue.cc:260] T 0d911577ce40499f93ab70176722159f P 6b51c58c505c44c7943c02c022ddfa18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:18.319249 21597 heartbeater.cc:503] Master 127.18.168.190:36557 was elected leader, sending a full tablet report...
I20260810 18:48:18.319532 21625 ts_tablet_manager.cc:1428] T 0d911577ce40499f93ab70176722159f P 6b51c58c505c44c7943c02c022ddfa18: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260810 18:48:18.319002 21622 raft_consensus.cc:348] T 0d911577ce40499f93ab70176722159f P 1c06a8266d164fdc98b6891041be38d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:18.319826 21622 raft_consensus.cc:374] T 0d911577ce40499f93ab70176722159f P 1c06a8266d164fdc98b6891041be38d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:18.319893 21622 raft_consensus.cc:729] T 0d911577ce40499f93ab70176722159f P 1c06a8266d164fdc98b6891041be38d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c06a8266d164fdc98b6891041be38d0, State: Initialized, Role: FOLLOWER
I20260810 18:48:18.320331 21622 consensus_queue.cc:260] T 0d911577ce40499f93ab70176722159f P 1c06a8266d164fdc98b6891041be38d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:18.321383 21622 ts_tablet_manager.cc:1428] T 0d911577ce40499f93ab70176722159f P 1c06a8266d164fdc98b6891041be38d0: Time spent starting tablet: real 0.007s user 0.005s sys 0.002s
I20260810 18:48:18.321580 21624 raft_consensus.cc:348] T 0d911577ce40499f93ab70176722159f P ec02cf09b8914f118a120895e3c0f3cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:18.321883 21624 raft_consensus.cc:374] T 0d911577ce40499f93ab70176722159f P ec02cf09b8914f118a120895e3c0f3cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:18.321939 21624 raft_consensus.cc:729] T 0d911577ce40499f93ab70176722159f P ec02cf09b8914f118a120895e3c0f3cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec02cf09b8914f118a120895e3c0f3cf, State: Initialized, Role: FOLLOWER
I20260810 18:48:18.322245 21436 heartbeater.cc:503] Master 127.18.168.190:36557 was elected leader, sending a full tablet report...
I20260810 18:48:18.322458 21624 consensus_queue.cc:260] T 0d911577ce40499f93ab70176722159f P ec02cf09b8914f118a120895e3c0f3cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:18.323148 21277 heartbeater.cc:503] Master 127.18.168.190:36557 was elected leader, sending a full tablet report...
I20260810 18:48:18.323447 21624 ts_tablet_manager.cc:1428] T 0d911577ce40499f93ab70176722159f P ec02cf09b8914f118a120895e3c0f3cf: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260810 18:48:18.374935 21629 raft_consensus.cc:482] T 0d911577ce40499f93ab70176722159f P 6b51c58c505c44c7943c02c022ddfa18 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:48:18.375173 21629 raft_consensus.cc:504] T 0d911577ce40499f93ab70176722159f P 6b51c58c505c44c7943c02c022ddfa18 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:18.376559 21629 leader_election.cc:288] T 0d911577ce40499f93ab70176722159f P 6b51c58c505c44c7943c02c022ddfa18 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec02cf09b8914f118a120895e3c0f3cf (127.18.168.129:41069), 1c06a8266d164fdc98b6891041be38d0 (127.18.168.130:35203)
I20260810 18:48:18.380273 21474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:18.376791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:41069 (local address 127.18.168.131:47653)
0810 18:48:18.376896 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:18.376913 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:18.377009 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:18.377382 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:18.377385 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:18.377406 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:18.377544 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:18.377572 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.378338 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.378343 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:18.379137 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:18.379146 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.379831 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.379835 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:18.379943 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:18.379971 (+ 28us) client_negotiation.cc:769] Sending connection context
0810 18:48:18.380061 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 18:48:18.380136 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"spinlock_wait_cycles":21760}
I20260810 18:48:18.380272 21618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:18.376957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:47653 (local address 127.18.168.129:41069)
0810 18:48:18.377088 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:18.377094 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:18.377109 (+ 15us) server_negotiation.cc:407] Connection header received
0810 18:48:18.377215 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:18.377217 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:18.377259 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:18.377364 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:18.377687 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.378213 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.379271 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.379678 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.379833 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:18.379887 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:18.380082 (+ 195us) server_negotiation.cc:299] Negotiation successful
0810 18:48:18.380162 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 18:48:18.380963 21222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d911577ce40499f93ab70176722159f" candidate_uuid: "6b51c58c505c44c7943c02c022ddfa18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec02cf09b8914f118a120895e3c0f3cf" is_pre_election: true
I20260810 18:48:18.381067 21620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:18.376958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:38249 (local address 127.18.168.130:35203)
0810 18:48:18.377076 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:18.377080 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:18.377393 (+ 313us) server_negotiation.cc:407] Connection header received
0810 18:48:18.377664 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:18.377674 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:18.377720 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:18.377800 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:18.378431 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.378934 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.379919 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.380340 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.380475 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:18.380520 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:18.380891 (+ 371us) server_negotiation.cc:299] Negotiation successful
0810 18:48:18.380965 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 18:48:18.381083 21632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:18.376791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:35203 (local address 127.18.168.131:38249)
0810 18:48:18.377291 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:18.377302 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:18.377477 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:18.377825 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:18.377827 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:18.377843 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:18.378303 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:18.378309 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.379038 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.379042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:18.379788 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:18.379796 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.380575 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.380579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:18.380736 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:18.380766 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 18:48:18.380872 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 18:48:18.380938 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"mutex_wait_us":113,"spinlock_wait_cycles":768,"thread_start_us":157,"threads_started":1}
I20260810 18:48:18.381317 21222 raft_consensus.cc:2454] T 0d911577ce40499f93ab70176722159f P ec02cf09b8914f118a120895e3c0f3cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b51c58c505c44c7943c02c022ddfa18 in term 0.
I20260810 18:48:18.381682 21382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d911577ce40499f93ab70176722159f" candidate_uuid: "6b51c58c505c44c7943c02c022ddfa18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c06a8266d164fdc98b6891041be38d0" is_pre_election: true
I20260810 18:48:18.381851 21465 leader_election.cc:302] T 0d911577ce40499f93ab70176722159f P 6b51c58c505c44c7943c02c022ddfa18 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b51c58c505c44c7943c02c022ddfa18, ec02cf09b8914f118a120895e3c0f3cf; no voters:
I20260810 18:48:18.381980 21382 raft_consensus.cc:2454] T 0d911577ce40499f93ab70176722159f P 1c06a8266d164fdc98b6891041be38d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b51c58c505c44c7943c02c022ddfa18 in term 0.
I20260810 18:48:18.382220 21629 raft_consensus.cc:2781] T 0d911577ce40499f93ab70176722159f P 6b51c58c505c44c7943c02c022ddfa18 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:48:18.382308 21629 raft_consensus.cc:482] T 0d911577ce40499f93ab70176722159f P 6b51c58c505c44c7943c02c022ddfa18 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:48:18.382366 21629 raft_consensus.cc:3037] T 0d911577ce40499f93ab70176722159f P 6b51c58c505c44c7943c02c022ddfa18 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:18.383459 21629 raft_consensus.cc:504] T 0d911577ce40499f93ab70176722159f P 6b51c58c505c44c7943c02c022ddfa18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:18.383839 21629 leader_election.cc:288] T 0d911577ce40499f93ab70176722159f P 6b51c58c505c44c7943c02c022ddfa18 [CANDIDATE]: Term 1 election: Requested vote from peers ec02cf09b8914f118a120895e3c0f3cf (127.18.168.129:41069), 1c06a8266d164fdc98b6891041be38d0 (127.18.168.130:35203)
I20260810 18:48:18.384189 21382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d911577ce40499f93ab70176722159f" candidate_uuid: "6b51c58c505c44c7943c02c022ddfa18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c06a8266d164fdc98b6891041be38d0"
I20260810 18:48:18.384244 21222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d911577ce40499f93ab70176722159f" candidate_uuid: "6b51c58c505c44c7943c02c022ddfa18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec02cf09b8914f118a120895e3c0f3cf"
I20260810 18:48:18.384331 21382 raft_consensus.cc:3037] T 0d911577ce40499f93ab70176722159f P 1c06a8266d164fdc98b6891041be38d0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:18.384367 21222 raft_consensus.cc:3037] T 0d911577ce40499f93ab70176722159f P ec02cf09b8914f118a120895e3c0f3cf [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:18.385440 21382 raft_consensus.cc:2454] T 0d911577ce40499f93ab70176722159f P 1c06a8266d164fdc98b6891041be38d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b51c58c505c44c7943c02c022ddfa18 in term 1.
I20260810 18:48:18.385501 21222 raft_consensus.cc:2454] T 0d911577ce40499f93ab70176722159f P ec02cf09b8914f118a120895e3c0f3cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b51c58c505c44c7943c02c022ddfa18 in term 1.
I20260810 18:48:18.385779 21467 leader_election.cc:302] T 0d911577ce40499f93ab70176722159f P 6b51c58c505c44c7943c02c022ddfa18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c06a8266d164fdc98b6891041be38d0, 6b51c58c505c44c7943c02c022ddfa18; no voters:
I20260810 18:48:18.385941 21629 raft_consensus.cc:2781] T 0d911577ce40499f93ab70176722159f P 6b51c58c505c44c7943c02c022ddfa18 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:18.386145 21629 raft_consensus.cc:686] T 0d911577ce40499f93ab70176722159f P 6b51c58c505c44c7943c02c022ddfa18 [term 1 LEADER]: Becoming Leader. State: Replica: 6b51c58c505c44c7943c02c022ddfa18, State: Running, Role: LEADER
I20260810 18:48:18.386487 21629 consensus_queue.cc:237] T 0d911577ce40499f93ab70176722159f P 6b51c58c505c44c7943c02c022ddfa18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:18.389916 21051 catalog_manager.cc:5495] T 0d911577ce40499f93ab70176722159f P 6b51c58c505c44c7943c02c022ddfa18 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b51c58c505c44c7943c02c022ddfa18 (127.18.168.131). New cstate: current_term: 1 leader_uuid: "6b51c58c505c44c7943c02c022ddfa18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } health_report { overall_health: HEALTHY } } }
Aug 10 18:48:18 dist-test-slave-cbsf krb5kdc[20998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387697, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:18.451189 21641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:18.442640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:36557 (local address 127.0.0.1:50050)
0810 18:48:18.442993 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:18.443006 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:18.443151 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:18.443421 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:18.443423 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:18.443851 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:18.443969 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:18.443979 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.444794 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.444797 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:18.445357 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:18.445365 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.445860 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.445863 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:18.446008 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:18.446547 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:18.446575 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:18.448013 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:18.449611 (+ 1598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:18.449622 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:18.449629 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:18.449887 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:18.450231 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:18.450234 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:18.450235 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:18.450318 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:18.450737 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:18.450743 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:18.450848 (+ 105us) client_negotiation.cc:769] Sending connection context
0810 18:48:18.450938 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 18:48:18.451062 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":39,"thread_start_us":135,"threads_started":1}
I20260810 18:48:18.451336 21106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:18.442703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50050 (local address 127.18.168.190:36557)
0810 18:48:18.442849 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:18.442857 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:18.443096 (+ 239us) server_negotiation.cc:407] Connection header received
0810 18:48:18.443254 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:18.443257 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:18.443303 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:18.443401 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:18.444110 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.444639 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.445489 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.445724 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.445845 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:18.448167 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:18.448172 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:18.448176 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:18.448202 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:18.449468 (+ 1266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:18.450007 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:18.450011 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:18.450013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:18.450111 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:18.450411 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:18.450414 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:18.450416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:18.450635 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:18.450734 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:18.451025 (+ 291us) server_negotiation.cc:299] Negotiation successful
0810 18:48:18.451158 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":47}
I20260810 18:48:18.455418 21051 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:50050:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 18:48:18.456199 21051 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:48:18.461282 21201 tablet_service.cc:1467] Processing CreateTablet for tablet 425a3cc32f434d93b7c49699460b6f1a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=201d80516e104fb9b0c4f3ca0d86f912]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 18:48:18.461287 21361 tablet_service.cc:1467] Processing CreateTablet for tablet 425a3cc32f434d93b7c49699460b6f1a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=201d80516e104fb9b0c4f3ca0d86f912]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 18:48:18.461570 21520 tablet_service.cc:1467] Processing CreateTablet for tablet 425a3cc32f434d93b7c49699460b6f1a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=201d80516e104fb9b0c4f3ca0d86f912]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 18:48:18.462042 21201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 425a3cc32f434d93b7c49699460b6f1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:18.462062 21361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 425a3cc32f434d93b7c49699460b6f1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:18.462139 21520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 425a3cc32f434d93b7c49699460b6f1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:18.465042 21624 tablet_bootstrap.cc:492] T 425a3cc32f434d93b7c49699460b6f1a P ec02cf09b8914f118a120895e3c0f3cf: Bootstrap starting.
I20260810 18:48:18.465317 21625 tablet_bootstrap.cc:492] T 425a3cc32f434d93b7c49699460b6f1a P 6b51c58c505c44c7943c02c022ddfa18: Bootstrap starting.
I20260810 18:48:18.465457 21622 tablet_bootstrap.cc:492] T 425a3cc32f434d93b7c49699460b6f1a P 1c06a8266d164fdc98b6891041be38d0: Bootstrap starting.
I20260810 18:48:18.466094 21624 tablet_bootstrap.cc:654] T 425a3cc32f434d93b7c49699460b6f1a P ec02cf09b8914f118a120895e3c0f3cf: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:18.466307 21625 tablet_bootstrap.cc:654] T 425a3cc32f434d93b7c49699460b6f1a P 6b51c58c505c44c7943c02c022ddfa18: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:18.466508 21622 tablet_bootstrap.cc:654] T 425a3cc32f434d93b7c49699460b6f1a P 1c06a8266d164fdc98b6891041be38d0: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:18.467655 21624 tablet_bootstrap.cc:492] T 425a3cc32f434d93b7c49699460b6f1a P ec02cf09b8914f118a120895e3c0f3cf: No bootstrap required, opened a new log
I20260810 18:48:18.467746 21624 ts_tablet_manager.cc:1397] T 425a3cc32f434d93b7c49699460b6f1a P ec02cf09b8914f118a120895e3c0f3cf: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 18:48:18.468304 21622 tablet_bootstrap.cc:492] T 425a3cc32f434d93b7c49699460b6f1a P 1c06a8266d164fdc98b6891041be38d0: No bootstrap required, opened a new log
I20260810 18:48:18.468304 21624 raft_consensus.cc:348] T 425a3cc32f434d93b7c49699460b6f1a P ec02cf09b8914f118a120895e3c0f3cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:18.468398 21622 ts_tablet_manager.cc:1397] T 425a3cc32f434d93b7c49699460b6f1a P 1c06a8266d164fdc98b6891041be38d0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 18:48:18.468432 21624 raft_consensus.cc:374] T 425a3cc32f434d93b7c49699460b6f1a P ec02cf09b8914f118a120895e3c0f3cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:18.468477 21624 raft_consensus.cc:729] T 425a3cc32f434d93b7c49699460b6f1a P ec02cf09b8914f118a120895e3c0f3cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec02cf09b8914f118a120895e3c0f3cf, State: Initialized, Role: FOLLOWER
I20260810 18:48:18.468649 21624 consensus_queue.cc:260] T 425a3cc32f434d93b7c49699460b6f1a P ec02cf09b8914f118a120895e3c0f3cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:18.468906 21622 raft_consensus.cc:348] T 425a3cc32f434d93b7c49699460b6f1a P 1c06a8266d164fdc98b6891041be38d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:18.469044 21622 raft_consensus.cc:374] T 425a3cc32f434d93b7c49699460b6f1a P 1c06a8266d164fdc98b6891041be38d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:18.469082 21622 raft_consensus.cc:729] T 425a3cc32f434d93b7c49699460b6f1a P 1c06a8266d164fdc98b6891041be38d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c06a8266d164fdc98b6891041be38d0, State: Initialized, Role: FOLLOWER
I20260810 18:48:18.468999 21631 tablet_replica.cc:439] T 425a3cc32f434d93b7c49699460b6f1a P ec02cf09b8914f118a120895e3c0f3cf: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } } }
I20260810 18:48:18.469203 21631 tablet_replica.cc:442] T 425a3cc32f434d93b7c49699460b6f1a P ec02cf09b8914f118a120895e3c0f3cf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:48:18.469204 21622 consensus_queue.cc:260] T 425a3cc32f434d93b7c49699460b6f1a P 1c06a8266d164fdc98b6891041be38d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:18.469002 21624 ts_tablet_manager.cc:1428] T 425a3cc32f434d93b7c49699460b6f1a P ec02cf09b8914f118a120895e3c0f3cf: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260810 18:48:18.469455 21622 ts_tablet_manager.cc:1428] T 425a3cc32f434d93b7c49699460b6f1a P 1c06a8266d164fdc98b6891041be38d0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260810 18:48:18.469475 21630 tablet_replica.cc:439] T 425a3cc32f434d93b7c49699460b6f1a P 1c06a8266d164fdc98b6891041be38d0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } } }
I20260810 18:48:18.469661 21630 tablet_replica.cc:442] T 425a3cc32f434d93b7c49699460b6f1a P 1c06a8266d164fdc98b6891041be38d0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:48:18.470433 21625 tablet_bootstrap.cc:492] T 425a3cc32f434d93b7c49699460b6f1a P 6b51c58c505c44c7943c02c022ddfa18: No bootstrap required, opened a new log
I20260810 18:48:18.470537 21625 ts_tablet_manager.cc:1397] T 425a3cc32f434d93b7c49699460b6f1a P 6b51c58c505c44c7943c02c022ddfa18: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20260810 18:48:18.470914 21625 raft_consensus.cc:348] T 425a3cc32f434d93b7c49699460b6f1a P 6b51c58c505c44c7943c02c022ddfa18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:18.471043 21625 raft_consensus.cc:374] T 425a3cc32f434d93b7c49699460b6f1a P 6b51c58c505c44c7943c02c022ddfa18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:18.471083 21625 raft_consensus.cc:729] T 425a3cc32f434d93b7c49699460b6f1a P 6b51c58c505c44c7943c02c022ddfa18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b51c58c505c44c7943c02c022ddfa18, State: Initialized, Role: FOLLOWER
I20260810 18:48:18.471213 21625 consensus_queue.cc:260] T 425a3cc32f434d93b7c49699460b6f1a P 6b51c58c505c44c7943c02c022ddfa18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:18.471506 21625 ts_tablet_manager.cc:1428] T 425a3cc32f434d93b7c49699460b6f1a P 6b51c58c505c44c7943c02c022ddfa18: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 18:48:18.471470 21629 tablet_replica.cc:439] T 425a3cc32f434d93b7c49699460b6f1a P 6b51c58c505c44c7943c02c022ddfa18: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } } }
I20260810 18:48:18.471596 21629 tablet_replica.cc:442] T 425a3cc32f434d93b7c49699460b6f1a P 6b51c58c505c44c7943c02c022ddfa18: This TxnStatusTablet replica's current role is: FOLLOWER
W20260810 18:48:18.485077 21598 tablet.cc:2369] T 0d911577ce40499f93ab70176722159f P 6b51c58c505c44c7943c02c022ddfa18: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:48:18.494632 21437 tablet.cc:2369] T 0d911577ce40499f93ab70176722159f P 1c06a8266d164fdc98b6891041be38d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:48:18.506983 21631 raft_consensus.cc:482] T 425a3cc32f434d93b7c49699460b6f1a P ec02cf09b8914f118a120895e3c0f3cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:48:18.507115 21631 raft_consensus.cc:504] T 425a3cc32f434d93b7c49699460b6f1a P ec02cf09b8914f118a120895e3c0f3cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:18.508284 21631 leader_election.cc:288] T 425a3cc32f434d93b7c49699460b6f1a P ec02cf09b8914f118a120895e3c0f3cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c06a8266d164fdc98b6891041be38d0 (127.18.168.130:35203), 6b51c58c505c44c7943c02c022ddfa18 (127.18.168.131:46491)
I20260810 18:48:18.513876 21620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:18.508672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:55177 (local address 127.18.168.130:35203)
0810 18:48:18.508858 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:18.508862 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:18.509061 (+ 199us) server_negotiation.cc:407] Connection header received
0810 18:48:18.509266 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:18.509270 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:18.509327 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:18.509402 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:18.509764 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.510549 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.512502 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.513149 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.513266 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:18.513333 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:18.513681 (+ 348us) server_negotiation.cc:299] Negotiation successful
0810 18:48:18.513749 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260810 18:48:18.513876 21621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:18.509210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:48217 (local address 127.18.168.131:46491)
0810 18:48:18.509339 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:18.509343 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:18.509510 (+ 167us) server_negotiation.cc:407] Connection header received
0810 18:48:18.509624 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:18.509626 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:18.509669 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:18.509765 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:18.510007 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.510756 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.512502 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.513161 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.513266 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:18.513319 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:18.513675 (+ 356us) server_negotiation.cc:299] Negotiation successful
0810 18:48:18.513749 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 18:48:18.513949 21646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:18.508594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:35203 (local address 127.18.168.129:55177)
0810 18:48:18.508952 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:18.508965 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:18.509133 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:18.509450 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:18.509456 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:18.509481 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:18.509643 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:18.509651 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.510660 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.510663 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:18.512366 (+ 1703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:18.512379 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.513298 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.513302 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:18.513447 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:18.513561 (+ 114us) client_negotiation.cc:769] Sending connection context
0810 18:48:18.513644 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 18:48:18.513738 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"mutex_wait_us":131,"spinlock_wait_cycles":2816,"thread_start_us":180,"threads_started":1}
I20260810 18:48:18.513950 21648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:18.509084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.131:46491 (local address 127.18.168.129:48217)
0810 18:48:18.509401 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:18.509416 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:18.509516 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:18.509791 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:18.509793 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:18.509810 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:18.509894 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:18.509899 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.510854 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.510859 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:18.512366 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:18.512379 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.513298 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.513302 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:18.513506 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:18.513563 (+ 57us) client_negotiation.cc:769] Sending connection context
0810 18:48:18.513644 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 18:48:18.513738 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":173,"spinlock_wait_cycles":896,"thread_start_us":102,"threads_started":1}
I20260810 18:48:18.514511 21382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "425a3cc32f434d93b7c49699460b6f1a" candidate_uuid: "ec02cf09b8914f118a120895e3c0f3cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c06a8266d164fdc98b6891041be38d0" is_pre_election: true
I20260810 18:48:18.514690 21382 raft_consensus.cc:2454] T 425a3cc32f434d93b7c49699460b6f1a P 1c06a8266d164fdc98b6891041be38d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec02cf09b8914f118a120895e3c0f3cf in term 0.
I20260810 18:48:18.514699 21542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "425a3cc32f434d93b7c49699460b6f1a" candidate_uuid: "ec02cf09b8914f118a120895e3c0f3cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b51c58c505c44c7943c02c022ddfa18" is_pre_election: true
I20260810 18:48:18.514958 21542 raft_consensus.cc:2454] T 425a3cc32f434d93b7c49699460b6f1a P 6b51c58c505c44c7943c02c022ddfa18 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec02cf09b8914f118a120895e3c0f3cf in term 0.
I20260810 18:48:18.515066 21152 leader_election.cc:302] T 425a3cc32f434d93b7c49699460b6f1a P ec02cf09b8914f118a120895e3c0f3cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c06a8266d164fdc98b6891041be38d0, ec02cf09b8914f118a120895e3c0f3cf; no voters:
I20260810 18:48:18.515440 21631 raft_consensus.cc:2781] T 425a3cc32f434d93b7c49699460b6f1a P ec02cf09b8914f118a120895e3c0f3cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:48:18.515555 21631 raft_consensus.cc:482] T 425a3cc32f434d93b7c49699460b6f1a P ec02cf09b8914f118a120895e3c0f3cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:48:18.515605 21631 raft_consensus.cc:3037] T 425a3cc32f434d93b7c49699460b6f1a P ec02cf09b8914f118a120895e3c0f3cf [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:18.516738 21631 raft_consensus.cc:504] T 425a3cc32f434d93b7c49699460b6f1a P ec02cf09b8914f118a120895e3c0f3cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:18.517153 21631 leader_election.cc:288] T 425a3cc32f434d93b7c49699460b6f1a P ec02cf09b8914f118a120895e3c0f3cf [CANDIDATE]: Term 1 election: Requested vote from peers 1c06a8266d164fdc98b6891041be38d0 (127.18.168.130:35203), 6b51c58c505c44c7943c02c022ddfa18 (127.18.168.131:46491)
I20260810 18:48:18.517553 21382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "425a3cc32f434d93b7c49699460b6f1a" candidate_uuid: "ec02cf09b8914f118a120895e3c0f3cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c06a8266d164fdc98b6891041be38d0"
I20260810 18:48:18.517697 21382 raft_consensus.cc:3037] T 425a3cc32f434d93b7c49699460b6f1a P 1c06a8266d164fdc98b6891041be38d0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:18.518576 21382 raft_consensus.cc:2454] T 425a3cc32f434d93b7c49699460b6f1a P 1c06a8266d164fdc98b6891041be38d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec02cf09b8914f118a120895e3c0f3cf in term 1.
I20260810 18:48:18.518872 21152 leader_election.cc:302] T 425a3cc32f434d93b7c49699460b6f1a P ec02cf09b8914f118a120895e3c0f3cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c06a8266d164fdc98b6891041be38d0, ec02cf09b8914f118a120895e3c0f3cf; no voters:
I20260810 18:48:18.518846 21542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "425a3cc32f434d93b7c49699460b6f1a" candidate_uuid: "ec02cf09b8914f118a120895e3c0f3cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b51c58c505c44c7943c02c022ddfa18"
I20260810 18:48:18.519016 21542 raft_consensus.cc:3037] T 425a3cc32f434d93b7c49699460b6f1a P 6b51c58c505c44c7943c02c022ddfa18 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:18.519049 21631 raft_consensus.cc:2781] T 425a3cc32f434d93b7c49699460b6f1a P ec02cf09b8914f118a120895e3c0f3cf [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:18.519263 21631 raft_consensus.cc:686] T 425a3cc32f434d93b7c49699460b6f1a P ec02cf09b8914f118a120895e3c0f3cf [term 1 LEADER]: Becoming Leader. State: Replica: ec02cf09b8914f118a120895e3c0f3cf, State: Running, Role: LEADER
I20260810 18:48:18.519641 21631 consensus_queue.cc:237] T 425a3cc32f434d93b7c49699460b6f1a P ec02cf09b8914f118a120895e3c0f3cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
W20260810 18:48:18.519865 21278 tablet.cc:2369] T 0d911577ce40499f93ab70176722159f P ec02cf09b8914f118a120895e3c0f3cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:48:18.520205 21542 raft_consensus.cc:2454] T 425a3cc32f434d93b7c49699460b6f1a P 6b51c58c505c44c7943c02c022ddfa18 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec02cf09b8914f118a120895e3c0f3cf in term 1.
I20260810 18:48:18.522248 21649 tablet_replica.cc:439] T 425a3cc32f434d93b7c49699460b6f1a P ec02cf09b8914f118a120895e3c0f3cf: TxnStatusTablet state changed. Reason: New leader ec02cf09b8914f118a120895e3c0f3cf. Latest consensus state: current_term: 1 leader_uuid: "ec02cf09b8914f118a120895e3c0f3cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } } }
I20260810 18:48:18.522360 21649 tablet_replica.cc:442] T 425a3cc32f434d93b7c49699460b6f1a P ec02cf09b8914f118a120895e3c0f3cf: This TxnStatusTablet replica's current role is: LEADER
I20260810 18:48:18.522616 21651 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 18:48:18.522838 21651 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 18:48:18.523167 21049 catalog_manager.cc:5495] T 425a3cc32f434d93b7c49699460b6f1a P ec02cf09b8914f118a120895e3c0f3cf reported cstate change: term changed from 0 to 1, leader changed from <none> to ec02cf09b8914f118a120895e3c0f3cf (127.18.168.129). New cstate: current_term: 1 leader_uuid: "ec02cf09b8914f118a120895e3c0f3cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } health_report { overall_health: UNKNOWN } } }
I20260810 18:48:18.540911 21641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:18.536433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:41069 (local address 127.0.0.1:38582)
0810 18:48:18.536617 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:18.536635 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:18.536804 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:18.537174 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:18.537177 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:18.537183 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:48:18.537339 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:18.537344 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.538394 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.538398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:18.539153 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:18.539163 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.539715 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.539718 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:18.539814 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:18.539861 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:48:18.540614 (+ 753us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:48:18.540628 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 18:48:18.540684 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 18:48:18.540806 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260810 18:48:18.540915 21618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:18.536580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38582 (local address 127.18.168.129:41069)
0810 18:48:18.536751 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:18.536755 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:18.536771 (+ 16us) server_negotiation.cc:407] Connection header received
0810 18:48:18.536943 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:18.536948 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:18.537014 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:18.537113 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:48:18.537442 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.538248 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.539274 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.539602 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.539706 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:18.540009 (+ 303us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:48:18.540519 (+ 510us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:48:18.540629 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:18.540718 (+ 89us) server_negotiation.cc:299] Negotiation successful
0810 18:48:18.540775 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 18:48:18.546016 21382 raft_consensus.cc:1261] T 425a3cc32f434d93b7c49699460b6f1a P 1c06a8266d164fdc98b6891041be38d0 [term 1 FOLLOWER]: Refusing update from remote peer ec02cf09b8914f118a120895e3c0f3cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:18.546016 21542 raft_consensus.cc:1261] T 425a3cc32f434d93b7c49699460b6f1a P 6b51c58c505c44c7943c02c022ddfa18 [term 1 FOLLOWER]: Refusing update from remote peer ec02cf09b8914f118a120895e3c0f3cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:18.546689 21649 consensus_queue.cc:1035] T 425a3cc32f434d93b7c49699460b6f1a P ec02cf09b8914f118a120895e3c0f3cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:18.546960 21631 consensus_queue.cc:1035] T 425a3cc32f434d93b7c49699460b6f1a P ec02cf09b8914f118a120895e3c0f3cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:18.550285 21629 tablet_replica.cc:439] T 425a3cc32f434d93b7c49699460b6f1a P 6b51c58c505c44c7943c02c022ddfa18: TxnStatusTablet state changed. Reason: New leader ec02cf09b8914f118a120895e3c0f3cf. Latest consensus state: current_term: 1 leader_uuid: "ec02cf09b8914f118a120895e3c0f3cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } } }
I20260810 18:48:18.550406 21629 tablet_replica.cc:442] T 425a3cc32f434d93b7c49699460b6f1a P 6b51c58c505c44c7943c02c022ddfa18: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:48:18.550931 21630 tablet_replica.cc:439] T 425a3cc32f434d93b7c49699460b6f1a P 1c06a8266d164fdc98b6891041be38d0: TxnStatusTablet state changed. Reason: New leader ec02cf09b8914f118a120895e3c0f3cf. Latest consensus state: current_term: 1 leader_uuid: "ec02cf09b8914f118a120895e3c0f3cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } } }
I20260810 18:48:18.551049 21630 tablet_replica.cc:442] T 425a3cc32f434d93b7c49699460b6f1a P 1c06a8266d164fdc98b6891041be38d0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:48:18.554029 21629 tablet_replica.cc:439] T 425a3cc32f434d93b7c49699460b6f1a P 6b51c58c505c44c7943c02c022ddfa18: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ec02cf09b8914f118a120895e3c0f3cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } } }
I20260810 18:48:18.554141 21629 tablet_replica.cc:442] T 425a3cc32f434d93b7c49699460b6f1a P 6b51c58c505c44c7943c02c022ddfa18: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:48:18.554088 21631 tablet_replica.cc:439] T 425a3cc32f434d93b7c49699460b6f1a P ec02cf09b8914f118a120895e3c0f3cf: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ec02cf09b8914f118a120895e3c0f3cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } } }
I20260810 18:48:18.554180 21631 tablet_replica.cc:442] T 425a3cc32f434d93b7c49699460b6f1a P ec02cf09b8914f118a120895e3c0f3cf: This TxnStatusTablet replica's current role is: LEADER
I20260810 18:48:18.554289 21631 tablet_replica.cc:439] T 425a3cc32f434d93b7c49699460b6f1a P ec02cf09b8914f118a120895e3c0f3cf: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ec02cf09b8914f118a120895e3c0f3cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } } }
I20260810 18:48:18.554361 21631 tablet_replica.cc:442] T 425a3cc32f434d93b7c49699460b6f1a P ec02cf09b8914f118a120895e3c0f3cf: This TxnStatusTablet replica's current role is: LEADER
I20260810 18:48:18.554498 21654 mvcc.cc:205] Tried to move back new op lower bound from 7317044013237379072 to 7317044013139816448. Current Snapshot: MvccSnapshot[applied={T|T < 7317044013237379072}]
I20260810 18:48:18.555526 21630 tablet_replica.cc:439] T 425a3cc32f434d93b7c49699460b6f1a P 1c06a8266d164fdc98b6891041be38d0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ec02cf09b8914f118a120895e3c0f3cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } } }
I20260810 18:48:18.555630 21630 tablet_replica.cc:442] T 425a3cc32f434d93b7c49699460b6f1a P 1c06a8266d164fdc98b6891041be38d0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:48:18.557508 21656 mvcc.cc:205] Tried to move back new op lower bound from 7317044013237379072 to 7317044013139816448. Current Snapshot: MvccSnapshot[applied={T|T < 7317044013237379072}]
I20260810 18:48:18.569767 21621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:18.565805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41538 (local address 127.18.168.131:46491)
0810 18:48:18.565920 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:18.565923 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:18.565938 (+ 15us) server_negotiation.cc:407] Connection header received
0810 18:48:18.566165 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:18.566168 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:18.566229 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:18.566345 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:48:18.566588 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.567202 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.568376 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.568605 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.568681 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:18.569000 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:48:18.569360 (+ 360us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:48:18.569440 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:18.569570 (+ 130us) server_negotiation.cc:299] Negotiation successful
0810 18:48:18.569616 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 18:48:18.580636 21604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:18.577266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:41069 (local address 127.18.168.131:43889)
0810 18:48:18.577401 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:18.577417 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:18.577493 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:18.577756 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:18.577758 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:18.577764 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:48:18.577848 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:18.577852 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.578844 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.578846 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:18.579410 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:18.579417 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.579982 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.579984 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:18.580064 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:18.580103 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:48:18.580439 (+ 336us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:48:18.580443 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 18:48:18.580490 (+ 47us) client_negotiation.cc:240] Negotiation successful
0810 18:48:18.580545 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260810 18:48:18.580739 21618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:18.577336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:43889 (local address 127.18.168.129:41069)
0810 18:48:18.577449 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:18.577454 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:18.577468 (+ 14us) server_negotiation.cc:407] Connection header received
0810 18:48:18.577595 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:18.577598 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:18.577654 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:18.577727 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:48:18.577943 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.578748 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.579515 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.579867 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.580000 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:18.580275 (+ 275us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:48:18.580354 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:48:18.580429 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:18.580500 (+ 71us) server_negotiation.cc:299] Negotiation successful
0810 18:48:18.580550 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 18:48:18.588804 21382 raft_consensus.cc:1261] T 0d911577ce40499f93ab70176722159f P 1c06a8266d164fdc98b6891041be38d0 [term 1 FOLLOWER]: Refusing update from remote peer 6b51c58c505c44c7943c02c022ddfa18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:18.589362 21629 consensus_queue.cc:1035] T 0d911577ce40499f93ab70176722159f P 6b51c58c505c44c7943c02c022ddfa18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:18.590229 21222 raft_consensus.cc:1261] T 0d911577ce40499f93ab70176722159f P ec02cf09b8914f118a120895e3c0f3cf [term 1 FOLLOWER]: Refusing update from remote peer 6b51c58c505c44c7943c02c022ddfa18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:18.590771 21629 consensus_queue.cc:1035] T 0d911577ce40499f93ab70176722159f P 6b51c58c505c44c7943c02c022ddfa18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:18.613327 21621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:18.608769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:38099 (local address 127.18.168.131:46491)
0810 18:48:18.608863 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:18.608867 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:18.609175 (+ 308us) server_negotiation.cc:407] Connection header received
0810 18:48:18.609328 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:18.609331 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:18.609375 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:18.609457 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:48:18.609838 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.610665 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.612016 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.612363 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.612471 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:18.612902 (+ 431us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:48:18.612994 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:48:18.613069 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:18.613147 (+ 78us) server_negotiation.cc:299] Negotiation successful
0810 18:48:18.613198 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 18:48:18.613370 21671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:18.608672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.131:46491 (local address 127.18.168.129:38099)
0810 18:48:18.609086 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:18.609098 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:18.609223 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:18.609479 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:18.609482 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:18.609489 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:48:18.609698 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:18.609711 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.610779 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.610782 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:18.611919 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:18.611929 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.612479 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.612482 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:18.612673 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:18.612769 (+ 96us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:48:18.613079 (+ 310us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:48:18.613085 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 18:48:18.613135 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 18:48:18.613195 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":161,"threads_started":1}
I20260810 18:48:18.641346 21049 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:18.641600 21049 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:18.644634 21049 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fbe4cd70c6a54bdb8500cd1b1369f5be: Sending DeleteTablet for 3 replicas of tablet 0d911577ce40499f93ab70176722159f
I20260810 18:48:18.645310 21201 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d911577ce40499f93ab70176722159f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:18 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:38568
I20260810 18:48:18.645733 21361 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d911577ce40499f93ab70176722159f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:18 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:48192
I20260810 18:48:18.646045 21674 tablet_replica.cc:330] T 0d911577ce40499f93ab70176722159f P ec02cf09b8914f118a120895e3c0f3cf: stopping tablet replica
I20260810 18:48:18.646358 21674 raft_consensus.cc:2229] T 0d911577ce40499f93ab70176722159f P ec02cf09b8914f118a120895e3c0f3cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:18.646523 21674 raft_consensus.cc:2258] T 0d911577ce40499f93ab70176722159f P ec02cf09b8914f118a120895e3c0f3cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:18.646593 21520 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d911577ce40499f93ab70176722159f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:18 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:41532
I20260810 18:48:18.647850 21674 ts_tablet_manager.cc:1905] T 0d911577ce40499f93ab70176722159f P ec02cf09b8914f118a120895e3c0f3cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:18.648025 21049 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50034:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:48:18.648449 21049 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:48:18.651607 21674 ts_tablet_manager.cc:1918] T 0d911577ce40499f93ab70176722159f P ec02cf09b8914f118a120895e3c0f3cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:48:18.651715 21674 log.cc:1196] T 0d911577ce40499f93ab70176722159f P ec02cf09b8914f118a120895e3c0f3cf: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-0/wal/wals/0d911577ce40499f93ab70176722159f
I20260810 18:48:18.652176 21674 ts_tablet_manager.cc:1939] T 0d911577ce40499f93ab70176722159f P ec02cf09b8914f118a120895e3c0f3cf: Deleting consensus metadata
I20260810 18:48:18.652343 21675 tablet_replica.cc:330] T 0d911577ce40499f93ab70176722159f P 1c06a8266d164fdc98b6891041be38d0: stopping tablet replica
I20260810 18:48:18.653679 21033 catalog_manager.cc:4841] TS ec02cf09b8914f118a120895e3c0f3cf (127.18.168.129:41069): tablet 0d911577ce40499f93ab70176722159f (table test-table [id=f925a305f263443fa86d3b9223916a52]) successfully deleted
I20260810 18:48:18.654018 21201 tablet_service.cc:1467] Processing CreateTablet for tablet f6b6f98684c44293930a7d11f2a8a226 (DEFAULT_TABLE table=test-table [id=6f70744481fa43d5946c73c1eab5f4ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:18.654273 21201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6b6f98684c44293930a7d11f2a8a226. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:18.654806 21361 tablet_service.cc:1467] Processing CreateTablet for tablet f6b6f98684c44293930a7d11f2a8a226 (DEFAULT_TABLE table=test-table [id=6f70744481fa43d5946c73c1eab5f4ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:18.655105 21361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6b6f98684c44293930a7d11f2a8a226. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:18.656864 21624 tablet_bootstrap.cc:492] T f6b6f98684c44293930a7d11f2a8a226 P ec02cf09b8914f118a120895e3c0f3cf: Bootstrap starting.
I20260810 18:48:18.657584 21622 tablet_bootstrap.cc:492] T f6b6f98684c44293930a7d11f2a8a226 P 1c06a8266d164fdc98b6891041be38d0: Bootstrap starting.
I20260810 18:48:18.657778 21520 tablet_service.cc:1467] Processing CreateTablet for tablet f6b6f98684c44293930a7d11f2a8a226 (DEFAULT_TABLE table=test-table [id=6f70744481fa43d5946c73c1eab5f4ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:18.657845 21624 tablet_bootstrap.cc:654] T f6b6f98684c44293930a7d11f2a8a226 P ec02cf09b8914f118a120895e3c0f3cf: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:18.658054 21520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6b6f98684c44293930a7d11f2a8a226. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:18.659161 21622 tablet_bootstrap.cc:654] T f6b6f98684c44293930a7d11f2a8a226 P 1c06a8266d164fdc98b6891041be38d0: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:18.660645 21625 tablet_bootstrap.cc:492] T f6b6f98684c44293930a7d11f2a8a226 P 6b51c58c505c44c7943c02c022ddfa18: Bootstrap starting.
I20260810 18:48:18.660830 21622 tablet_bootstrap.cc:492] T f6b6f98684c44293930a7d11f2a8a226 P 1c06a8266d164fdc98b6891041be38d0: No bootstrap required, opened a new log
I20260810 18:48:18.660917 21622 ts_tablet_manager.cc:1397] T f6b6f98684c44293930a7d11f2a8a226 P 1c06a8266d164fdc98b6891041be38d0: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.002s
I20260810 18:48:18.661378 21622 raft_consensus.cc:348] T f6b6f98684c44293930a7d11f2a8a226 P 1c06a8266d164fdc98b6891041be38d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:18.661499 21622 raft_consensus.cc:374] T f6b6f98684c44293930a7d11f2a8a226 P 1c06a8266d164fdc98b6891041be38d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:18.661538 21622 raft_consensus.cc:729] T f6b6f98684c44293930a7d11f2a8a226 P 1c06a8266d164fdc98b6891041be38d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c06a8266d164fdc98b6891041be38d0, State: Initialized, Role: FOLLOWER
I20260810 18:48:18.661564 21625 tablet_bootstrap.cc:654] T f6b6f98684c44293930a7d11f2a8a226 P 6b51c58c505c44c7943c02c022ddfa18: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:18.661669 21622 consensus_queue.cc:260] T f6b6f98684c44293930a7d11f2a8a226 P 1c06a8266d164fdc98b6891041be38d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:18.661960 21622 ts_tablet_manager.cc:1428] T f6b6f98684c44293930a7d11f2a8a226 P 1c06a8266d164fdc98b6891041be38d0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 18:48:18.663039 21676 tablet_replica.cc:330] T 0d911577ce40499f93ab70176722159f P 6b51c58c505c44c7943c02c022ddfa18: stopping tablet replica
I20260810 18:48:18.663383 21676 raft_consensus.cc:2229] T 0d911577ce40499f93ab70176722159f P 6b51c58c505c44c7943c02c022ddfa18 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:48:18.663975 21675 raft_consensus.cc:2229] T 0d911577ce40499f93ab70176722159f P 1c06a8266d164fdc98b6891041be38d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:18.664559 21676 raft_consensus.cc:2258] T 0d911577ce40499f93ab70176722159f P 6b51c58c505c44c7943c02c022ddfa18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:18.664819 21675 raft_consensus.cc:2258] T 0d911577ce40499f93ab70176722159f P 1c06a8266d164fdc98b6891041be38d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:18.666340 21675 ts_tablet_manager.cc:1905] T 0d911577ce40499f93ab70176722159f P 1c06a8266d164fdc98b6891041be38d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:18.669389 21675 ts_tablet_manager.cc:1918] T 0d911577ce40499f93ab70176722159f P 1c06a8266d164fdc98b6891041be38d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:48:18.669502 21675 log.cc:1196] T 0d911577ce40499f93ab70176722159f P 1c06a8266d164fdc98b6891041be38d0: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-1/wal/wals/0d911577ce40499f93ab70176722159f
I20260810 18:48:18.669627 21624 tablet_bootstrap.cc:492] T f6b6f98684c44293930a7d11f2a8a226 P ec02cf09b8914f118a120895e3c0f3cf: No bootstrap required, opened a new log
I20260810 18:48:18.669759 21624 ts_tablet_manager.cc:1397] T f6b6f98684c44293930a7d11f2a8a226 P ec02cf09b8914f118a120895e3c0f3cf: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20260810 18:48:18.669847 21675 ts_tablet_manager.cc:1939] T 0d911577ce40499f93ab70176722159f P 1c06a8266d164fdc98b6891041be38d0: Deleting consensus metadata
I20260810 18:48:18.670239 21624 raft_consensus.cc:348] T f6b6f98684c44293930a7d11f2a8a226 P ec02cf09b8914f118a120895e3c0f3cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:18.670378 21624 raft_consensus.cc:374] T f6b6f98684c44293930a7d11f2a8a226 P ec02cf09b8914f118a120895e3c0f3cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:18.670423 21624 raft_consensus.cc:729] T f6b6f98684c44293930a7d11f2a8a226 P ec02cf09b8914f118a120895e3c0f3cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec02cf09b8914f118a120895e3c0f3cf, State: Initialized, Role: FOLLOWER
I20260810 18:48:18.670558 21624 consensus_queue.cc:260] T f6b6f98684c44293930a7d11f2a8a226 P ec02cf09b8914f118a120895e3c0f3cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:18.670837 21624 ts_tablet_manager.cc:1428] T f6b6f98684c44293930a7d11f2a8a226 P ec02cf09b8914f118a120895e3c0f3cf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 18:48:18.671064 21035 catalog_manager.cc:4841] TS 1c06a8266d164fdc98b6891041be38d0 (127.18.168.130:35203): tablet 0d911577ce40499f93ab70176722159f (table test-table [id=f925a305f263443fa86d3b9223916a52]) successfully deleted
I20260810 18:48:18.672101 21676 ts_tablet_manager.cc:1905] T 0d911577ce40499f93ab70176722159f P 6b51c58c505c44c7943c02c022ddfa18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:18.674948 21676 ts_tablet_manager.cc:1918] T 0d911577ce40499f93ab70176722159f P 6b51c58c505c44c7943c02c022ddfa18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:48:18.675379 21676 log.cc:1196] T 0d911577ce40499f93ab70176722159f P 6b51c58c505c44c7943c02c022ddfa18: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-2/wal/wals/0d911577ce40499f93ab70176722159f
I20260810 18:48:18.675714 21676 ts_tablet_manager.cc:1939] T 0d911577ce40499f93ab70176722159f P 6b51c58c505c44c7943c02c022ddfa18: Deleting consensus metadata
I20260810 18:48:18.675803 21625 tablet_bootstrap.cc:492] T f6b6f98684c44293930a7d11f2a8a226 P 6b51c58c505c44c7943c02c022ddfa18: No bootstrap required, opened a new log
I20260810 18:48:18.675889 21625 ts_tablet_manager.cc:1397] T f6b6f98684c44293930a7d11f2a8a226 P 6b51c58c505c44c7943c02c022ddfa18: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.002s
I20260810 18:48:18.676369 21625 raft_consensus.cc:348] T f6b6f98684c44293930a7d11f2a8a226 P 6b51c58c505c44c7943c02c022ddfa18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:18.676492 21625 raft_consensus.cc:374] T f6b6f98684c44293930a7d11f2a8a226 P 6b51c58c505c44c7943c02c022ddfa18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:18.676529 21625 raft_consensus.cc:729] T f6b6f98684c44293930a7d11f2a8a226 P 6b51c58c505c44c7943c02c022ddfa18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b51c58c505c44c7943c02c022ddfa18, State: Initialized, Role: FOLLOWER
I20260810 18:48:18.676674 21625 consensus_queue.cc:260] T f6b6f98684c44293930a7d11f2a8a226 P 6b51c58c505c44c7943c02c022ddfa18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:18.676870 21035 catalog_manager.cc:4841] TS 6b51c58c505c44c7943c02c022ddfa18 (127.18.168.131:46491): tablet 0d911577ce40499f93ab70176722159f (table test-table [id=f925a305f263443fa86d3b9223916a52]) successfully deleted
I20260810 18:48:18.676998 21625 ts_tablet_manager.cc:1428] T f6b6f98684c44293930a7d11f2a8a226 P 6b51c58c505c44c7943c02c022ddfa18: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260810 18:48:18.741021 21666 raft_consensus.cc:482] T f6b6f98684c44293930a7d11f2a8a226 P ec02cf09b8914f118a120895e3c0f3cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:48:18.741214 21666 raft_consensus.cc:504] T f6b6f98684c44293930a7d11f2a8a226 P ec02cf09b8914f118a120895e3c0f3cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:18.741654 21666 leader_election.cc:288] T f6b6f98684c44293930a7d11f2a8a226 P ec02cf09b8914f118a120895e3c0f3cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c06a8266d164fdc98b6891041be38d0 (127.18.168.130:35203), 6b51c58c505c44c7943c02c022ddfa18 (127.18.168.131:46491)
I20260810 18:48:18.742084 21542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6b6f98684c44293930a7d11f2a8a226" candidate_uuid: "ec02cf09b8914f118a120895e3c0f3cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b51c58c505c44c7943c02c022ddfa18" is_pre_election: true
I20260810 18:48:18.742123 21382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6b6f98684c44293930a7d11f2a8a226" candidate_uuid: "ec02cf09b8914f118a120895e3c0f3cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c06a8266d164fdc98b6891041be38d0" is_pre_election: true
I20260810 18:48:18.742267 21542 raft_consensus.cc:2454] T f6b6f98684c44293930a7d11f2a8a226 P 6b51c58c505c44c7943c02c022ddfa18 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec02cf09b8914f118a120895e3c0f3cf in term 0.
I20260810 18:48:18.742290 21382 raft_consensus.cc:2454] T f6b6f98684c44293930a7d11f2a8a226 P 1c06a8266d164fdc98b6891041be38d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec02cf09b8914f118a120895e3c0f3cf in term 0.
I20260810 18:48:18.742627 21152 leader_election.cc:302] T f6b6f98684c44293930a7d11f2a8a226 P ec02cf09b8914f118a120895e3c0f3cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c06a8266d164fdc98b6891041be38d0, ec02cf09b8914f118a120895e3c0f3cf; no voters:
I20260810 18:48:18.742864 21666 raft_consensus.cc:2781] T f6b6f98684c44293930a7d11f2a8a226 P ec02cf09b8914f118a120895e3c0f3cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:48:18.742952 21666 raft_consensus.cc:482] T f6b6f98684c44293930a7d11f2a8a226 P ec02cf09b8914f118a120895e3c0f3cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:48:18.743007 21666 raft_consensus.cc:3037] T f6b6f98684c44293930a7d11f2a8a226 P ec02cf09b8914f118a120895e3c0f3cf [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:18.744066 21666 raft_consensus.cc:504] T f6b6f98684c44293930a7d11f2a8a226 P ec02cf09b8914f118a120895e3c0f3cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:18.744462 21666 leader_election.cc:288] T f6b6f98684c44293930a7d11f2a8a226 P ec02cf09b8914f118a120895e3c0f3cf [CANDIDATE]: Term 1 election: Requested vote from peers 1c06a8266d164fdc98b6891041be38d0 (127.18.168.130:35203), 6b51c58c505c44c7943c02c022ddfa18 (127.18.168.131:46491)
I20260810 18:48:18.744959 21382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6b6f98684c44293930a7d11f2a8a226" candidate_uuid: "ec02cf09b8914f118a120895e3c0f3cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c06a8266d164fdc98b6891041be38d0"
I20260810 18:48:18.744977 21542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6b6f98684c44293930a7d11f2a8a226" candidate_uuid: "ec02cf09b8914f118a120895e3c0f3cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b51c58c505c44c7943c02c022ddfa18"
I20260810 18:48:18.745092 21542 raft_consensus.cc:3037] T f6b6f98684c44293930a7d11f2a8a226 P 6b51c58c505c44c7943c02c022ddfa18 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:18.745100 21382 raft_consensus.cc:3037] T f6b6f98684c44293930a7d11f2a8a226 P 1c06a8266d164fdc98b6891041be38d0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:18.746181 21382 raft_consensus.cc:2454] T f6b6f98684c44293930a7d11f2a8a226 P 1c06a8266d164fdc98b6891041be38d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec02cf09b8914f118a120895e3c0f3cf in term 1.
I20260810 18:48:18.746173 21542 raft_consensus.cc:2454] T f6b6f98684c44293930a7d11f2a8a226 P 6b51c58c505c44c7943c02c022ddfa18 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec02cf09b8914f118a120895e3c0f3cf in term 1.
I20260810 18:48:18.746513 21152 leader_election.cc:302] T f6b6f98684c44293930a7d11f2a8a226 P ec02cf09b8914f118a120895e3c0f3cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c06a8266d164fdc98b6891041be38d0, ec02cf09b8914f118a120895e3c0f3cf; no voters:
I20260810 18:48:18.746809 21666 raft_consensus.cc:2781] T f6b6f98684c44293930a7d11f2a8a226 P ec02cf09b8914f118a120895e3c0f3cf [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:18.746941 21666 raft_consensus.cc:686] T f6b6f98684c44293930a7d11f2a8a226 P ec02cf09b8914f118a120895e3c0f3cf [term 1 LEADER]: Becoming Leader. State: Replica: ec02cf09b8914f118a120895e3c0f3cf, State: Running, Role: LEADER
I20260810 18:48:18.747079 21666 consensus_queue.cc:237] T f6b6f98684c44293930a7d11f2a8a226 P ec02cf09b8914f118a120895e3c0f3cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:18.749117 21049 catalog_manager.cc:5495] T f6b6f98684c44293930a7d11f2a8a226 P ec02cf09b8914f118a120895e3c0f3cf reported cstate change: term changed from 0 to 1, leader changed from <none> to ec02cf09b8914f118a120895e3c0f3cf (127.18.168.129). New cstate: current_term: 1 leader_uuid: "ec02cf09b8914f118a120895e3c0f3cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } health_report { overall_health: UNKNOWN } } }
I20260810 18:48:18.775180 21618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:18.771221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38590 (local address 127.18.168.129:41069)
0810 18:48:18.771329 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:18.771334 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:18.771349 (+ 15us) server_negotiation.cc:407] Connection header received
0810 18:48:18.771404 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:18.771406 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:18.771451 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:18.771526 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:48:18.771820 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.772567 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.773957 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.774293 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.774388 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:18.774698 (+ 310us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:48:18.774776 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:48:18.774839 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:18.775010 (+ 171us) server_negotiation.cc:299] Negotiation successful
0810 18:48:18.775058 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 18:48:18.785315 21671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:18.781351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:41069 (local address 127.18.168.129:53705)
0810 18:48:18.781475 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:18.781488 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:18.781558 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:18.781869 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:18.781872 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:18.781878 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:48:18.781971 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:18.781976 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.783014 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.783017 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:18.783916 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:18.783925 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.784550 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.784553 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:18.784638 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:18.784667 (+ 29us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:48:18.784985 (+ 318us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:48:18.784991 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 18:48:18.785055 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 18:48:18.785129 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260810 18:48:18.785319 21618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:18.781544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:53705 (local address 127.18.168.129:41069)
0810 18:48:18.781641 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:18.781646 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:18.781659 (+ 13us) server_negotiation.cc:407] Connection header received
0810 18:48:18.781699 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:18.781702 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:18.781746 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:18.781848 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:48:18.782094 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.782889 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.784069 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.784427 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.784545 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:18.784802 (+ 257us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:48:18.784885 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:48:18.784967 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:18.785066 (+ 99us) server_negotiation.cc:299] Negotiation successful
0810 18:48:18.785138 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 18:48:18.792147 21382 raft_consensus.cc:1261] T f6b6f98684c44293930a7d11f2a8a226 P 1c06a8266d164fdc98b6891041be38d0 [term 1 FOLLOWER]: Refusing update from remote peer ec02cf09b8914f118a120895e3c0f3cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:18.792378 21542 raft_consensus.cc:1261] T f6b6f98684c44293930a7d11f2a8a226 P 6b51c58c505c44c7943c02c022ddfa18 [term 1 FOLLOWER]: Refusing update from remote peer ec02cf09b8914f118a120895e3c0f3cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:18.792601 21649 consensus_queue.cc:1035] T f6b6f98684c44293930a7d11f2a8a226 P ec02cf09b8914f118a120895e3c0f3cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:18.792835 21631 consensus_queue.cc:1035] T f6b6f98684c44293930a7d11f2a8a226 P ec02cf09b8914f118a120895e3c0f3cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 18:48:18.808369 21654 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 18:48:20.375018 21049 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:20.375164 21049 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:20.377951 21049 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fbe4cd70c6a54bdb8500cd1b1369f5be: Sending DeleteTablet for 3 replicas of tablet f6b6f98684c44293930a7d11f2a8a226
I20260810 18:48:20.378525 21201 tablet_service.cc:1514] Processing DeleteTablet for tablet f6b6f98684c44293930a7d11f2a8a226 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:20 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:38568
I20260810 18:48:20.378633 21361 tablet_service.cc:1514] Processing DeleteTablet for tablet f6b6f98684c44293930a7d11f2a8a226 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:20 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:48192
I20260810 18:48:20.378832 21520 tablet_service.cc:1514] Processing DeleteTablet for tablet f6b6f98684c44293930a7d11f2a8a226 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:20 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:41532
I20260810 18:48:20.379002 21710 tablet_replica.cc:330] T f6b6f98684c44293930a7d11f2a8a226 P 1c06a8266d164fdc98b6891041be38d0: stopping tablet replica
I20260810 18:48:20.379172 21708 tablet_replica.cc:330] T f6b6f98684c44293930a7d11f2a8a226 P ec02cf09b8914f118a120895e3c0f3cf: stopping tablet replica
I20260810 18:48:20.379201 21710 raft_consensus.cc:2229] T f6b6f98684c44293930a7d11f2a8a226 P 1c06a8266d164fdc98b6891041be38d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:20.379357 21710 raft_consensus.cc:2258] T f6b6f98684c44293930a7d11f2a8a226 P 1c06a8266d164fdc98b6891041be38d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:20.379377 21708 raft_consensus.cc:2229] T f6b6f98684c44293930a7d11f2a8a226 P ec02cf09b8914f118a120895e3c0f3cf [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:48:20.379958 21708 raft_consensus.cc:2258] T f6b6f98684c44293930a7d11f2a8a226 P ec02cf09b8914f118a120895e3c0f3cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:20.380124 21710 ts_tablet_manager.cc:1905] T f6b6f98684c44293930a7d11f2a8a226 P 1c06a8266d164fdc98b6891041be38d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:20.380626 21708 ts_tablet_manager.cc:1905] T f6b6f98684c44293930a7d11f2a8a226 P ec02cf09b8914f118a120895e3c0f3cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:20.382849 21711 tablet_replica.cc:330] T f6b6f98684c44293930a7d11f2a8a226 P 6b51c58c505c44c7943c02c022ddfa18: stopping tablet replica
I20260810 18:48:20.382979 21711 raft_consensus.cc:2229] T f6b6f98684c44293930a7d11f2a8a226 P 6b51c58c505c44c7943c02c022ddfa18 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:20.383088 21711 raft_consensus.cc:2258] T f6b6f98684c44293930a7d11f2a8a226 P 6b51c58c505c44c7943c02c022ddfa18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:20.383684 21710 ts_tablet_manager.cc:1918] T f6b6f98684c44293930a7d11f2a8a226 P 1c06a8266d164fdc98b6891041be38d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 18:48:20.383701 21711 ts_tablet_manager.cc:1905] T f6b6f98684c44293930a7d11f2a8a226 P 6b51c58c505c44c7943c02c022ddfa18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:20.383793 21710 log.cc:1196] T f6b6f98684c44293930a7d11f2a8a226 P 1c06a8266d164fdc98b6891041be38d0: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-1/wal/wals/f6b6f98684c44293930a7d11f2a8a226
I20260810 18:48:20.384006 21708 ts_tablet_manager.cc:1918] T f6b6f98684c44293930a7d11f2a8a226 P ec02cf09b8914f118a120895e3c0f3cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 18:48:20.384124 21708 log.cc:1196] T f6b6f98684c44293930a7d11f2a8a226 P ec02cf09b8914f118a120895e3c0f3cf: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-0/wal/wals/f6b6f98684c44293930a7d11f2a8a226
I20260810 18:48:20.384188 21710 ts_tablet_manager.cc:1939] T f6b6f98684c44293930a7d11f2a8a226 P 1c06a8266d164fdc98b6891041be38d0: Deleting consensus metadata
I20260810 18:48:20.384438 21708 ts_tablet_manager.cc:1939] T f6b6f98684c44293930a7d11f2a8a226 P ec02cf09b8914f118a120895e3c0f3cf: Deleting consensus metadata
I20260810 18:48:20.384948 21035 catalog_manager.cc:4841] TS 1c06a8266d164fdc98b6891041be38d0 (127.18.168.130:35203): tablet f6b6f98684c44293930a7d11f2a8a226 (table test-table [id=6f70744481fa43d5946c73c1eab5f4ff]) successfully deleted
I20260810 18:48:20.386919 21711 ts_tablet_manager.cc:1918] T f6b6f98684c44293930a7d11f2a8a226 P 6b51c58c505c44c7943c02c022ddfa18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 18:48:20.387010 21711 log.cc:1196] T f6b6f98684c44293930a7d11f2a8a226 P 6b51c58c505c44c7943c02c022ddfa18: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-2/wal/wals/f6b6f98684c44293930a7d11f2a8a226
I20260810 18:48:20.387300 21711 ts_tablet_manager.cc:1939] T f6b6f98684c44293930a7d11f2a8a226 P 6b51c58c505c44c7943c02c022ddfa18: Deleting consensus metadata
I20260810 18:48:20.388156 21033 catalog_manager.cc:4841] TS ec02cf09b8914f118a120895e3c0f3cf (127.18.168.129:41069): tablet f6b6f98684c44293930a7d11f2a8a226 (table test-table [id=6f70744481fa43d5946c73c1eab5f4ff]) successfully deleted
I20260810 18:48:20.388213 21035 catalog_manager.cc:4841] TS 6b51c58c505c44c7943c02c022ddfa18 (127.18.168.131:46491): tablet f6b6f98684c44293930a7d11f2a8a226 (table test-table [id=6f70744481fa43d5946c73c1eab5f4ff]) successfully deleted
Aug 10 18:48:20 dist-test-slave-cbsf krb5kdc[20998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387700, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 18:48:20 dist-test-slave-cbsf krb5kdc[20998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387700, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:20.410908 21721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:20.402203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44844 (local address 127.18.168.190:36557)
0810 18:48:20.402574 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:20.402585 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:20.402603 (+ 18us) server_negotiation.cc:407] Connection header received
0810 18:48:20.402668 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:20.402671 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:20.402714 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:20.402814 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:20.403550 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:20.404098 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:20.405075 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:20.405330 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:20.405480 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:20.407794 (+ 2314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:20.407797 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:20.407799 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:20.407822 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:20.409198 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:20.409605 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:20.409610 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:20.409612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:20.409705 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:20.410003 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:20.410007 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:20.410008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:20.410202 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:20.410348 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:20.410594 (+ 246us) server_negotiation.cc:299] Negotiation successful
0810 18:48:20.410733 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":168,"threads_started":1}
I20260810 18:48:20.414001 21049 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44844:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:48:20.414536 21049 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:48:20.419636 21201 tablet_service.cc:1467] Processing CreateTablet for tablet 52cc299840d04fd6b27c4e3dc381d41e (DEFAULT_TABLE table=test-table [id=ba87045da2914bed83628d732827eaf6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:20.419791 21361 tablet_service.cc:1467] Processing CreateTablet for tablet 52cc299840d04fd6b27c4e3dc381d41e (DEFAULT_TABLE table=test-table [id=ba87045da2914bed83628d732827eaf6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:20.419843 21520 tablet_service.cc:1467] Processing CreateTablet for tablet 52cc299840d04fd6b27c4e3dc381d41e (DEFAULT_TABLE table=test-table [id=ba87045da2914bed83628d732827eaf6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:20.419992 21201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52cc299840d04fd6b27c4e3dc381d41e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:20.420091 21361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52cc299840d04fd6b27c4e3dc381d41e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:20.420261 21520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52cc299840d04fd6b27c4e3dc381d41e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:20.422633 21723 tablet_bootstrap.cc:492] T 52cc299840d04fd6b27c4e3dc381d41e P 1c06a8266d164fdc98b6891041be38d0: Bootstrap starting.
I20260810 18:48:20.422901 21722 tablet_bootstrap.cc:492] T 52cc299840d04fd6b27c4e3dc381d41e P ec02cf09b8914f118a120895e3c0f3cf: Bootstrap starting.
I20260810 18:48:20.423089 21725 tablet_bootstrap.cc:492] T 52cc299840d04fd6b27c4e3dc381d41e P 6b51c58c505c44c7943c02c022ddfa18: Bootstrap starting.
I20260810 18:48:20.424036 21722 tablet_bootstrap.cc:654] T 52cc299840d04fd6b27c4e3dc381d41e P ec02cf09b8914f118a120895e3c0f3cf: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:20.424064 21725 tablet_bootstrap.cc:654] T 52cc299840d04fd6b27c4e3dc381d41e P 6b51c58c505c44c7943c02c022ddfa18: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:20.425331 21722 tablet_bootstrap.cc:492] T 52cc299840d04fd6b27c4e3dc381d41e P ec02cf09b8914f118a120895e3c0f3cf: No bootstrap required, opened a new log
I20260810 18:48:20.425331 21725 tablet_bootstrap.cc:492] T 52cc299840d04fd6b27c4e3dc381d41e P 6b51c58c505c44c7943c02c022ddfa18: No bootstrap required, opened a new log
I20260810 18:48:20.425421 21722 ts_tablet_manager.cc:1397] T 52cc299840d04fd6b27c4e3dc381d41e P ec02cf09b8914f118a120895e3c0f3cf: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 18:48:20.425421 21725 ts_tablet_manager.cc:1397] T 52cc299840d04fd6b27c4e3dc381d41e P 6b51c58c505c44c7943c02c022ddfa18: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 18:48:20.425807 21725 raft_consensus.cc:348] T 52cc299840d04fd6b27c4e3dc381d41e P 6b51c58c505c44c7943c02c022ddfa18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:20.425938 21725 raft_consensus.cc:374] T 52cc299840d04fd6b27c4e3dc381d41e P 6b51c58c505c44c7943c02c022ddfa18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:20.425978 21725 raft_consensus.cc:729] T 52cc299840d04fd6b27c4e3dc381d41e P 6b51c58c505c44c7943c02c022ddfa18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b51c58c505c44c7943c02c022ddfa18, State: Initialized, Role: FOLLOWER
I20260810 18:48:20.426079 21723 tablet_bootstrap.cc:654] T 52cc299840d04fd6b27c4e3dc381d41e P 1c06a8266d164fdc98b6891041be38d0: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:20.426134 21725 consensus_queue.cc:260] T 52cc299840d04fd6b27c4e3dc381d41e P 6b51c58c505c44c7943c02c022ddfa18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:20.426530 21725 ts_tablet_manager.cc:1428] T 52cc299840d04fd6b27c4e3dc381d41e P 6b51c58c505c44c7943c02c022ddfa18: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 18:48:20.426612 21722 raft_consensus.cc:348] T 52cc299840d04fd6b27c4e3dc381d41e P ec02cf09b8914f118a120895e3c0f3cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:20.426710 21722 raft_consensus.cc:374] T 52cc299840d04fd6b27c4e3dc381d41e P ec02cf09b8914f118a120895e3c0f3cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:20.426745 21722 raft_consensus.cc:729] T 52cc299840d04fd6b27c4e3dc381d41e P ec02cf09b8914f118a120895e3c0f3cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec02cf09b8914f118a120895e3c0f3cf, State: Initialized, Role: FOLLOWER
I20260810 18:48:20.426882 21722 consensus_queue.cc:260] T 52cc299840d04fd6b27c4e3dc381d41e P ec02cf09b8914f118a120895e3c0f3cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:20.427174 21722 ts_tablet_manager.cc:1428] T 52cc299840d04fd6b27c4e3dc381d41e P ec02cf09b8914f118a120895e3c0f3cf: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260810 18:48:20.428251 21723 tablet_bootstrap.cc:492] T 52cc299840d04fd6b27c4e3dc381d41e P 1c06a8266d164fdc98b6891041be38d0: No bootstrap required, opened a new log
I20260810 18:48:20.428433 21723 ts_tablet_manager.cc:1397] T 52cc299840d04fd6b27c4e3dc381d41e P 1c06a8266d164fdc98b6891041be38d0: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260810 18:48:20.429445 21723 raft_consensus.cc:348] T 52cc299840d04fd6b27c4e3dc381d41e P 1c06a8266d164fdc98b6891041be38d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:20.429663 21723 raft_consensus.cc:374] T 52cc299840d04fd6b27c4e3dc381d41e P 1c06a8266d164fdc98b6891041be38d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:20.429766 21723 raft_consensus.cc:729] T 52cc299840d04fd6b27c4e3dc381d41e P 1c06a8266d164fdc98b6891041be38d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c06a8266d164fdc98b6891041be38d0, State: Initialized, Role: FOLLOWER
I20260810 18:48:20.429962 21723 consensus_queue.cc:260] T 52cc299840d04fd6b27c4e3dc381d41e P 1c06a8266d164fdc98b6891041be38d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:20.430421 21723 ts_tablet_manager.cc:1428] T 52cc299840d04fd6b27c4e3dc381d41e P 1c06a8266d164fdc98b6891041be38d0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 18:48:20.644878 21694 raft_consensus.cc:482] T 52cc299840d04fd6b27c4e3dc381d41e P ec02cf09b8914f118a120895e3c0f3cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:48:20.645040 21694 raft_consensus.cc:504] T 52cc299840d04fd6b27c4e3dc381d41e P ec02cf09b8914f118a120895e3c0f3cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:20.645484 21694 leader_election.cc:288] T 52cc299840d04fd6b27c4e3dc381d41e P ec02cf09b8914f118a120895e3c0f3cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c06a8266d164fdc98b6891041be38d0 (127.18.168.130:35203), 6b51c58c505c44c7943c02c022ddfa18 (127.18.168.131:46491)
I20260810 18:48:20.646010 21382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52cc299840d04fd6b27c4e3dc381d41e" candidate_uuid: "ec02cf09b8914f118a120895e3c0f3cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c06a8266d164fdc98b6891041be38d0" is_pre_election: true
I20260810 18:48:20.646103 21542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52cc299840d04fd6b27c4e3dc381d41e" candidate_uuid: "ec02cf09b8914f118a120895e3c0f3cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b51c58c505c44c7943c02c022ddfa18" is_pre_election: true
I20260810 18:48:20.646212 21382 raft_consensus.cc:2454] T 52cc299840d04fd6b27c4e3dc381d41e P 1c06a8266d164fdc98b6891041be38d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec02cf09b8914f118a120895e3c0f3cf in term 0.
I20260810 18:48:20.646276 21542 raft_consensus.cc:2454] T 52cc299840d04fd6b27c4e3dc381d41e P 6b51c58c505c44c7943c02c022ddfa18 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec02cf09b8914f118a120895e3c0f3cf in term 0.
I20260810 18:48:20.646643 21152 leader_election.cc:302] T 52cc299840d04fd6b27c4e3dc381d41e P ec02cf09b8914f118a120895e3c0f3cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c06a8266d164fdc98b6891041be38d0, ec02cf09b8914f118a120895e3c0f3cf; no voters:
I20260810 18:48:20.646843 21694 raft_consensus.cc:2781] T 52cc299840d04fd6b27c4e3dc381d41e P ec02cf09b8914f118a120895e3c0f3cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:48:20.646903 21694 raft_consensus.cc:482] T 52cc299840d04fd6b27c4e3dc381d41e P ec02cf09b8914f118a120895e3c0f3cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:48:20.646952 21694 raft_consensus.cc:3037] T 52cc299840d04fd6b27c4e3dc381d41e P ec02cf09b8914f118a120895e3c0f3cf [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:20.647980 21694 raft_consensus.cc:504] T 52cc299840d04fd6b27c4e3dc381d41e P ec02cf09b8914f118a120895e3c0f3cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:20.648295 21694 leader_election.cc:288] T 52cc299840d04fd6b27c4e3dc381d41e P ec02cf09b8914f118a120895e3c0f3cf [CANDIDATE]: Term 1 election: Requested vote from peers 1c06a8266d164fdc98b6891041be38d0 (127.18.168.130:35203), 6b51c58c505c44c7943c02c022ddfa18 (127.18.168.131:46491)
I20260810 18:48:20.648693 21382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52cc299840d04fd6b27c4e3dc381d41e" candidate_uuid: "ec02cf09b8914f118a120895e3c0f3cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c06a8266d164fdc98b6891041be38d0"
I20260810 18:48:20.648752 21542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52cc299840d04fd6b27c4e3dc381d41e" candidate_uuid: "ec02cf09b8914f118a120895e3c0f3cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b51c58c505c44c7943c02c022ddfa18"
I20260810 18:48:20.648836 21382 raft_consensus.cc:3037] T 52cc299840d04fd6b27c4e3dc381d41e P 1c06a8266d164fdc98b6891041be38d0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:20.648913 21542 raft_consensus.cc:3037] T 52cc299840d04fd6b27c4e3dc381d41e P 6b51c58c505c44c7943c02c022ddfa18 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:20.649832 21542 raft_consensus.cc:2454] T 52cc299840d04fd6b27c4e3dc381d41e P 6b51c58c505c44c7943c02c022ddfa18 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec02cf09b8914f118a120895e3c0f3cf in term 1.
I20260810 18:48:20.649871 21382 raft_consensus.cc:2454] T 52cc299840d04fd6b27c4e3dc381d41e P 1c06a8266d164fdc98b6891041be38d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec02cf09b8914f118a120895e3c0f3cf in term 1.
I20260810 18:48:20.650128 21152 leader_election.cc:302] T 52cc299840d04fd6b27c4e3dc381d41e P ec02cf09b8914f118a120895e3c0f3cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b51c58c505c44c7943c02c022ddfa18, ec02cf09b8914f118a120895e3c0f3cf; no voters:
I20260810 18:48:20.650288 21694 raft_consensus.cc:2781] T 52cc299840d04fd6b27c4e3dc381d41e P ec02cf09b8914f118a120895e3c0f3cf [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:20.650374 21694 raft_consensus.cc:686] T 52cc299840d04fd6b27c4e3dc381d41e P ec02cf09b8914f118a120895e3c0f3cf [term 1 LEADER]: Becoming Leader. State: Replica: ec02cf09b8914f118a120895e3c0f3cf, State: Running, Role: LEADER
I20260810 18:48:20.650517 21694 consensus_queue.cc:237] T 52cc299840d04fd6b27c4e3dc381d41e P ec02cf09b8914f118a120895e3c0f3cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:20.652305 21051 catalog_manager.cc:5495] T 52cc299840d04fd6b27c4e3dc381d41e P ec02cf09b8914f118a120895e3c0f3cf reported cstate change: term changed from 0 to 1, leader changed from <none> to ec02cf09b8914f118a120895e3c0f3cf (127.18.168.129). New cstate: current_term: 1 leader_uuid: "ec02cf09b8914f118a120895e3c0f3cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } health_report { overall_health: UNKNOWN } } }
I20260810 18:48:20.668682 21736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:20.665584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49416 (local address 127.18.168.129:41069)
0810 18:48:20.665866 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:20.665872 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:20.665884 (+ 12us) server_negotiation.cc:407] Connection header received
0810 18:48:20.665920 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:20.665923 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:20.665968 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:20.666038 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:48:20.666310 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:20.666924 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:20.667747 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:20.667985 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:20.668063 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:20.668300 (+ 237us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:48:20.668369 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:48:20.668423 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:20.668531 (+ 108us) server_negotiation.cc:299] Negotiation successful
0810 18:48:20.668574 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":146,"threads_started":1}
I20260810 18:48:20.674691 21542 raft_consensus.cc:1261] T 52cc299840d04fd6b27c4e3dc381d41e P 6b51c58c505c44c7943c02c022ddfa18 [term 1 FOLLOWER]: Refusing update from remote peer ec02cf09b8914f118a120895e3c0f3cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:20.674777 21382 raft_consensus.cc:1261] T 52cc299840d04fd6b27c4e3dc381d41e P 1c06a8266d164fdc98b6891041be38d0 [term 1 FOLLOWER]: Refusing update from remote peer ec02cf09b8914f118a120895e3c0f3cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:20.675114 21733 consensus_queue.cc:1035] T 52cc299840d04fd6b27c4e3dc381d41e P ec02cf09b8914f118a120895e3c0f3cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:20.675289 21689 consensus_queue.cc:1035] T 52cc299840d04fd6b27c4e3dc381d41e P ec02cf09b8914f118a120895e3c0f3cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:20.681373 21690 mvcc.cc:205] Tried to move back new op lower bound from 7317044021960491008 to 7317044021865889792. Current Snapshot: MvccSnapshot[applied={T|T < 7317044021960491008}]
I20260810 18:48:20.706108 21050 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:20.706261 21050 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:20.708114 21050 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fbe4cd70c6a54bdb8500cd1b1369f5be: Sending DeleteTablet for 3 replicas of tablet 52cc299840d04fd6b27c4e3dc381d41e
I20260810 18:48:20.708719 21201 tablet_service.cc:1514] Processing DeleteTablet for tablet 52cc299840d04fd6b27c4e3dc381d41e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:20 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:38568
I20260810 18:48:20.708781 21361 tablet_service.cc:1514] Processing DeleteTablet for tablet 52cc299840d04fd6b27c4e3dc381d41e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:20 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:48192
I20260810 18:48:20.708858 21520 tablet_service.cc:1514] Processing DeleteTablet for tablet 52cc299840d04fd6b27c4e3dc381d41e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:20 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:41532
I20260810 18:48:20.708983 21708 tablet_replica.cc:330] T 52cc299840d04fd6b27c4e3dc381d41e P ec02cf09b8914f118a120895e3c0f3cf: stopping tablet replica
I20260810 18:48:20.708988 21710 tablet_replica.cc:330] T 52cc299840d04fd6b27c4e3dc381d41e P 1c06a8266d164fdc98b6891041be38d0: stopping tablet replica
I20260810 18:48:20.709009 21711 tablet_replica.cc:330] T 52cc299840d04fd6b27c4e3dc381d41e P 6b51c58c505c44c7943c02c022ddfa18: stopping tablet replica
I20260810 18:48:20.709100 21711 raft_consensus.cc:2229] T 52cc299840d04fd6b27c4e3dc381d41e P 6b51c58c505c44c7943c02c022ddfa18 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:20.709095 21708 raft_consensus.cc:2229] T 52cc299840d04fd6b27c4e3dc381d41e P ec02cf09b8914f118a120895e3c0f3cf [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:48:20.709110 21710 raft_consensus.cc:2229] T 52cc299840d04fd6b27c4e3dc381d41e P 1c06a8266d164fdc98b6891041be38d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:20.709192 21711 raft_consensus.cc:2258] T 52cc299840d04fd6b27c4e3dc381d41e P 6b51c58c505c44c7943c02c022ddfa18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:20.709244 21710 raft_consensus.cc:2258] T 52cc299840d04fd6b27c4e3dc381d41e P 1c06a8266d164fdc98b6891041be38d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:20.709275 21708 raft_consensus.cc:2258] T 52cc299840d04fd6b27c4e3dc381d41e P ec02cf09b8914f118a120895e3c0f3cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:20.709703 21050 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44844:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 18:48:20.710021 21711 ts_tablet_manager.cc:1905] T 52cc299840d04fd6b27c4e3dc381d41e P 6b51c58c505c44c7943c02c022ddfa18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:20.710032 21710 ts_tablet_manager.cc:1905] T 52cc299840d04fd6b27c4e3dc381d41e P 1c06a8266d164fdc98b6891041be38d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:20.710045 21708 ts_tablet_manager.cc:1905] T 52cc299840d04fd6b27c4e3dc381d41e P ec02cf09b8914f118a120895e3c0f3cf: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 18:48:20.710108 21050 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:48:20.712484 21708 ts_tablet_manager.cc:1918] T 52cc299840d04fd6b27c4e3dc381d41e P ec02cf09b8914f118a120895e3c0f3cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:48:20.712711 21710 ts_tablet_manager.cc:1918] T 52cc299840d04fd6b27c4e3dc381d41e P 1c06a8266d164fdc98b6891041be38d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:48:20.712829 21710 log.cc:1196] T 52cc299840d04fd6b27c4e3dc381d41e P 1c06a8266d164fdc98b6891041be38d0: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-1/wal/wals/52cc299840d04fd6b27c4e3dc381d41e
I20260810 18:48:20.712884 21708 log.cc:1196] T 52cc299840d04fd6b27c4e3dc381d41e P ec02cf09b8914f118a120895e3c0f3cf: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-0/wal/wals/52cc299840d04fd6b27c4e3dc381d41e
I20260810 18:48:20.713169 21710 ts_tablet_manager.cc:1939] T 52cc299840d04fd6b27c4e3dc381d41e P 1c06a8266d164fdc98b6891041be38d0: Deleting consensus metadata
I20260810 18:48:20.713208 21708 ts_tablet_manager.cc:1939] T 52cc299840d04fd6b27c4e3dc381d41e P ec02cf09b8914f118a120895e3c0f3cf: Deleting consensus metadata
I20260810 18:48:20.712744 21711 ts_tablet_manager.cc:1918] T 52cc299840d04fd6b27c4e3dc381d41e P 6b51c58c505c44c7943c02c022ddfa18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:48:20.713487 21711 log.cc:1196] T 52cc299840d04fd6b27c4e3dc381d41e P 6b51c58c505c44c7943c02c022ddfa18: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-2/wal/wals/52cc299840d04fd6b27c4e3dc381d41e
I20260810 18:48:20.713789 21711 ts_tablet_manager.cc:1939] T 52cc299840d04fd6b27c4e3dc381d41e P 6b51c58c505c44c7943c02c022ddfa18: Deleting consensus metadata
I20260810 18:48:20.714159 21033 catalog_manager.cc:4841] TS ec02cf09b8914f118a120895e3c0f3cf (127.18.168.129:41069): tablet 52cc299840d04fd6b27c4e3dc381d41e (table test-table [id=ba87045da2914bed83628d732827eaf6]) successfully deleted
I20260810 18:48:20.714344 21035 catalog_manager.cc:4841] TS 1c06a8266d164fdc98b6891041be38d0 (127.18.168.130:35203): tablet 52cc299840d04fd6b27c4e3dc381d41e (table test-table [id=ba87045da2914bed83628d732827eaf6]) successfully deleted
I20260810 18:48:20.714471 21035 catalog_manager.cc:4841] TS 6b51c58c505c44c7943c02c022ddfa18 (127.18.168.131:46491): tablet 52cc299840d04fd6b27c4e3dc381d41e (table test-table [id=ba87045da2914bed83628d732827eaf6]) successfully deleted
I20260810 18:48:20.717095 21201 tablet_service.cc:1467] Processing CreateTablet for tablet 442d90cc310a4627a462c94f969c1782 (DEFAULT_TABLE table=test-table [id=5b15d56085334d17b8bacd1fbd386587]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:20.717309 21361 tablet_service.cc:1467] Processing CreateTablet for tablet 442d90cc310a4627a462c94f969c1782 (DEFAULT_TABLE table=test-table [id=5b15d56085334d17b8bacd1fbd386587]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:20.717403 21201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 442d90cc310a4627a462c94f969c1782. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:20.717568 21361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 442d90cc310a4627a462c94f969c1782. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:20.717571 21520 tablet_service.cc:1467] Processing CreateTablet for tablet 442d90cc310a4627a462c94f969c1782 (DEFAULT_TABLE table=test-table [id=5b15d56085334d17b8bacd1fbd386587]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:20.717805 21520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 442d90cc310a4627a462c94f969c1782. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:20.719295 21722 tablet_bootstrap.cc:492] T 442d90cc310a4627a462c94f969c1782 P ec02cf09b8914f118a120895e3c0f3cf: Bootstrap starting.
I20260810 18:48:20.719939 21723 tablet_bootstrap.cc:492] T 442d90cc310a4627a462c94f969c1782 P 1c06a8266d164fdc98b6891041be38d0: Bootstrap starting.
I20260810 18:48:20.720216 21722 tablet_bootstrap.cc:654] T 442d90cc310a4627a462c94f969c1782 P ec02cf09b8914f118a120895e3c0f3cf: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:20.720814 21725 tablet_bootstrap.cc:492] T 442d90cc310a4627a462c94f969c1782 P 6b51c58c505c44c7943c02c022ddfa18: Bootstrap starting.
I20260810 18:48:20.720961 21723 tablet_bootstrap.cc:654] T 442d90cc310a4627a462c94f969c1782 P 1c06a8266d164fdc98b6891041be38d0: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:20.721642 21725 tablet_bootstrap.cc:654] T 442d90cc310a4627a462c94f969c1782 P 6b51c58c505c44c7943c02c022ddfa18: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:20.722201 21723 tablet_bootstrap.cc:492] T 442d90cc310a4627a462c94f969c1782 P 1c06a8266d164fdc98b6891041be38d0: No bootstrap required, opened a new log
I20260810 18:48:20.722324 21723 ts_tablet_manager.cc:1397] T 442d90cc310a4627a462c94f969c1782 P 1c06a8266d164fdc98b6891041be38d0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 18:48:20.722590 21722 tablet_bootstrap.cc:492] T 442d90cc310a4627a462c94f969c1782 P ec02cf09b8914f118a120895e3c0f3cf: No bootstrap required, opened a new log
I20260810 18:48:20.722711 21722 ts_tablet_manager.cc:1397] T 442d90cc310a4627a462c94f969c1782 P ec02cf09b8914f118a120895e3c0f3cf: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 18:48:20.722803 21723 raft_consensus.cc:348] T 442d90cc310a4627a462c94f969c1782 P 1c06a8266d164fdc98b6891041be38d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:20.722920 21723 raft_consensus.cc:374] T 442d90cc310a4627a462c94f969c1782 P 1c06a8266d164fdc98b6891041be38d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:20.722952 21723 raft_consensus.cc:729] T 442d90cc310a4627a462c94f969c1782 P 1c06a8266d164fdc98b6891041be38d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c06a8266d164fdc98b6891041be38d0, State: Initialized, Role: FOLLOWER
I20260810 18:48:20.723096 21723 consensus_queue.cc:260] T 442d90cc310a4627a462c94f969c1782 P 1c06a8266d164fdc98b6891041be38d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:20.723199 21722 raft_consensus.cc:348] T 442d90cc310a4627a462c94f969c1782 P ec02cf09b8914f118a120895e3c0f3cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:20.723301 21722 raft_consensus.cc:374] T 442d90cc310a4627a462c94f969c1782 P ec02cf09b8914f118a120895e3c0f3cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:20.723343 21722 raft_consensus.cc:729] T 442d90cc310a4627a462c94f969c1782 P ec02cf09b8914f118a120895e3c0f3cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec02cf09b8914f118a120895e3c0f3cf, State: Initialized, Role: FOLLOWER
I20260810 18:48:20.723443 21723 ts_tablet_manager.cc:1428] T 442d90cc310a4627a462c94f969c1782 P 1c06a8266d164fdc98b6891041be38d0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 18:48:20.723488 21722 consensus_queue.cc:260] T 442d90cc310a4627a462c94f969c1782 P ec02cf09b8914f118a120895e3c0f3cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:20.723769 21722 ts_tablet_manager.cc:1428] T 442d90cc310a4627a462c94f969c1782 P ec02cf09b8914f118a120895e3c0f3cf: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 18:48:20.724223 21725 tablet_bootstrap.cc:492] T 442d90cc310a4627a462c94f969c1782 P 6b51c58c505c44c7943c02c022ddfa18: No bootstrap required, opened a new log
I20260810 18:48:20.724292 21725 ts_tablet_manager.cc:1397] T 442d90cc310a4627a462c94f969c1782 P 6b51c58c505c44c7943c02c022ddfa18: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260810 18:48:20.724558 21725 raft_consensus.cc:348] T 442d90cc310a4627a462c94f969c1782 P 6b51c58c505c44c7943c02c022ddfa18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:20.724639 21725 raft_consensus.cc:374] T 442d90cc310a4627a462c94f969c1782 P 6b51c58c505c44c7943c02c022ddfa18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:20.724725 21725 raft_consensus.cc:729] T 442d90cc310a4627a462c94f969c1782 P 6b51c58c505c44c7943c02c022ddfa18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b51c58c505c44c7943c02c022ddfa18, State: Initialized, Role: FOLLOWER
I20260810 18:48:20.724870 21725 consensus_queue.cc:260] T 442d90cc310a4627a462c94f969c1782 P 6b51c58c505c44c7943c02c022ddfa18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:20.725112 21725 ts_tablet_manager.cc:1428] T 442d90cc310a4627a462c94f969c1782 P 6b51c58c505c44c7943c02c022ddfa18: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260810 18:48:20.751194 21729 raft_consensus.cc:482] T 442d90cc310a4627a462c94f969c1782 P 6b51c58c505c44c7943c02c022ddfa18 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:48:20.751333 21729 raft_consensus.cc:504] T 442d90cc310a4627a462c94f969c1782 P 6b51c58c505c44c7943c02c022ddfa18 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:20.751703 21729 leader_election.cc:288] T 442d90cc310a4627a462c94f969c1782 P 6b51c58c505c44c7943c02c022ddfa18 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec02cf09b8914f118a120895e3c0f3cf (127.18.168.129:41069), 1c06a8266d164fdc98b6891041be38d0 (127.18.168.130:35203)
I20260810 18:48:20.752151 21382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "442d90cc310a4627a462c94f969c1782" candidate_uuid: "6b51c58c505c44c7943c02c022ddfa18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c06a8266d164fdc98b6891041be38d0" is_pre_election: true
I20260810 18:48:20.752163 21222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "442d90cc310a4627a462c94f969c1782" candidate_uuid: "6b51c58c505c44c7943c02c022ddfa18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec02cf09b8914f118a120895e3c0f3cf" is_pre_election: true
I20260810 18:48:20.752313 21382 raft_consensus.cc:2454] T 442d90cc310a4627a462c94f969c1782 P 1c06a8266d164fdc98b6891041be38d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b51c58c505c44c7943c02c022ddfa18 in term 0.
I20260810 18:48:20.752353 21222 raft_consensus.cc:2454] T 442d90cc310a4627a462c94f969c1782 P ec02cf09b8914f118a120895e3c0f3cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b51c58c505c44c7943c02c022ddfa18 in term 0.
I20260810 18:48:20.752737 21467 leader_election.cc:302] T 442d90cc310a4627a462c94f969c1782 P 6b51c58c505c44c7943c02c022ddfa18 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c06a8266d164fdc98b6891041be38d0, 6b51c58c505c44c7943c02c022ddfa18; no voters:
I20260810 18:48:20.752990 21729 raft_consensus.cc:2781] T 442d90cc310a4627a462c94f969c1782 P 6b51c58c505c44c7943c02c022ddfa18 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:48:20.753096 21729 raft_consensus.cc:482] T 442d90cc310a4627a462c94f969c1782 P 6b51c58c505c44c7943c02c022ddfa18 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:48:20.753154 21729 raft_consensus.cc:3037] T 442d90cc310a4627a462c94f969c1782 P 6b51c58c505c44c7943c02c022ddfa18 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:20.754302 21729 raft_consensus.cc:504] T 442d90cc310a4627a462c94f969c1782 P 6b51c58c505c44c7943c02c022ddfa18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:20.754626 21729 leader_election.cc:288] T 442d90cc310a4627a462c94f969c1782 P 6b51c58c505c44c7943c02c022ddfa18 [CANDIDATE]: Term 1 election: Requested vote from peers ec02cf09b8914f118a120895e3c0f3cf (127.18.168.129:41069), 1c06a8266d164fdc98b6891041be38d0 (127.18.168.130:35203)
I20260810 18:48:20.754940 21222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "442d90cc310a4627a462c94f969c1782" candidate_uuid: "6b51c58c505c44c7943c02c022ddfa18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec02cf09b8914f118a120895e3c0f3cf"
I20260810 18:48:20.755000 21382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "442d90cc310a4627a462c94f969c1782" candidate_uuid: "6b51c58c505c44c7943c02c022ddfa18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c06a8266d164fdc98b6891041be38d0"
I20260810 18:48:20.755092 21382 raft_consensus.cc:3037] T 442d90cc310a4627a462c94f969c1782 P 1c06a8266d164fdc98b6891041be38d0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:20.755092 21222 raft_consensus.cc:3037] T 442d90cc310a4627a462c94f969c1782 P ec02cf09b8914f118a120895e3c0f3cf [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:20.756199 21382 raft_consensus.cc:2454] T 442d90cc310a4627a462c94f969c1782 P 1c06a8266d164fdc98b6891041be38d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b51c58c505c44c7943c02c022ddfa18 in term 1.
I20260810 18:48:20.756191 21222 raft_consensus.cc:2454] T 442d90cc310a4627a462c94f969c1782 P ec02cf09b8914f118a120895e3c0f3cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b51c58c505c44c7943c02c022ddfa18 in term 1.
I20260810 18:48:20.756453 21465 leader_election.cc:302] T 442d90cc310a4627a462c94f969c1782 P 6b51c58c505c44c7943c02c022ddfa18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b51c58c505c44c7943c02c022ddfa18, ec02cf09b8914f118a120895e3c0f3cf; no voters:
I20260810 18:48:20.756628 21729 raft_consensus.cc:2781] T 442d90cc310a4627a462c94f969c1782 P 6b51c58c505c44c7943c02c022ddfa18 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:20.756883 21729 raft_consensus.cc:686] T 442d90cc310a4627a462c94f969c1782 P 6b51c58c505c44c7943c02c022ddfa18 [term 1 LEADER]: Becoming Leader. State: Replica: 6b51c58c505c44c7943c02c022ddfa18, State: Running, Role: LEADER
I20260810 18:48:20.757045 21729 consensus_queue.cc:237] T 442d90cc310a4627a462c94f969c1782 P 6b51c58c505c44c7943c02c022ddfa18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } }
I20260810 18:48:20.758678 21051 catalog_manager.cc:5495] T 442d90cc310a4627a462c94f969c1782 P 6b51c58c505c44c7943c02c022ddfa18 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b51c58c505c44c7943c02c022ddfa18 (127.18.168.131). New cstate: current_term: 1 leader_uuid: "6b51c58c505c44c7943c02c022ddfa18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b51c58c505c44c7943c02c022ddfa18" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 46491 } health_report { overall_health: HEALTHY } } }
I20260810 18:48:20.774121 21748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:20.770616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59094 (local address 127.18.168.131:46491)
0810 18:48:20.770913 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:20.770920 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:20.770931 (+ 11us) server_negotiation.cc:407] Connection header received
0810 18:48:20.770969 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:20.770971 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:20.771009 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:20.771077 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:48:20.771368 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:20.771882 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:20.773082 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:20.773304 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:20.773377 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:20.773652 (+ 275us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:48:20.773737 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:48:20.773816 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:20.773899 (+ 83us) server_negotiation.cc:299] Negotiation successful
0810 18:48:20.773943 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":158,"threads_started":1}
I20260810 18:48:20.780428 21222 raft_consensus.cc:1261] T 442d90cc310a4627a462c94f969c1782 P ec02cf09b8914f118a120895e3c0f3cf [term 1 FOLLOWER]: Refusing update from remote peer 6b51c58c505c44c7943c02c022ddfa18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:20.780411 21382 raft_consensus.cc:1261] T 442d90cc310a4627a462c94f969c1782 P 1c06a8266d164fdc98b6891041be38d0 [term 1 FOLLOWER]: Refusing update from remote peer 6b51c58c505c44c7943c02c022ddfa18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:20.780838 21729 consensus_queue.cc:1035] T 442d90cc310a4627a462c94f969c1782 P 6b51c58c505c44c7943c02c022ddfa18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec02cf09b8914f118a120895e3c0f3cf" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 41069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:20.781019 21746 consensus_queue.cc:1035] T 442d90cc310a4627a462c94f969c1782 P 6b51c58c505c44c7943c02c022ddfa18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c06a8266d164fdc98b6891041be38d0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 18:48:20.793866 21691 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 18:48:22.342798 21049 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:22.342939 21049 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:22.345292 21049 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fbe4cd70c6a54bdb8500cd1b1369f5be: Sending DeleteTablet for 3 replicas of tablet 442d90cc310a4627a462c94f969c1782
I20260810 18:48:22.345981 21201 tablet_service.cc:1514] Processing DeleteTablet for tablet 442d90cc310a4627a462c94f969c1782 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:22 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:38568
I20260810 18:48:22.345991 21520 tablet_service.cc:1514] Processing DeleteTablet for tablet 442d90cc310a4627a462c94f969c1782 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:22 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:41532
I20260810 18:48:22.346333 21777 tablet_replica.cc:330] T 442d90cc310a4627a462c94f969c1782 P 6b51c58c505c44c7943c02c022ddfa18: stopping tablet replica
I20260810 18:48:22.346338 21778 tablet_replica.cc:330] T 442d90cc310a4627a462c94f969c1782 P ec02cf09b8914f118a120895e3c0f3cf: stopping tablet replica
I20260810 18:48:22.346371 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 21130
I20260810 18:48:22.346939 21361 tablet_service.cc:1514] Processing DeleteTablet for tablet 442d90cc310a4627a462c94f969c1782 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:22 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:48192
I20260810 18:48:22.349653 21779 tablet_replica.cc:330] T 442d90cc310a4627a462c94f969c1782 P 1c06a8266d164fdc98b6891041be38d0: stopping tablet replica
I20260810 18:48:22.349793 21779 raft_consensus.cc:2229] T 442d90cc310a4627a462c94f969c1782 P 1c06a8266d164fdc98b6891041be38d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:22.349905 21779 raft_consensus.cc:2258] T 442d90cc310a4627a462c94f969c1782 P 1c06a8266d164fdc98b6891041be38d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:22.350070 21777 raft_consensus.cc:2229] T 442d90cc310a4627a462c94f969c1782 P 6b51c58c505c44c7943c02c022ddfa18 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:48:22.350355 21777 raft_consensus.cc:2258] T 442d90cc310a4627a462c94f969c1782 P 6b51c58c505c44c7943c02c022ddfa18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:22.350601 21779 ts_tablet_manager.cc:1905] T 442d90cc310a4627a462c94f969c1782 P 1c06a8266d164fdc98b6891041be38d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:22.351100 21777 ts_tablet_manager.cc:1905] T 442d90cc310a4627a462c94f969c1782 P 6b51c58c505c44c7943c02c022ddfa18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:22.353322 21779 ts_tablet_manager.cc:1918] T 442d90cc310a4627a462c94f969c1782 P 1c06a8266d164fdc98b6891041be38d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 18:48:22.353405 21779 log.cc:1196] T 442d90cc310a4627a462c94f969c1782 P 1c06a8266d164fdc98b6891041be38d0: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-1/wal/wals/442d90cc310a4627a462c94f969c1782
I20260810 18:48:22.353700 21779 ts_tablet_manager.cc:1939] T 442d90cc310a4627a462c94f969c1782 P 1c06a8266d164fdc98b6891041be38d0: Deleting consensus metadata
I20260810 18:48:22.353874 21777 ts_tablet_manager.cc:1918] T 442d90cc310a4627a462c94f969c1782 P 6b51c58c505c44c7943c02c022ddfa18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 18:48:22.353951 21777 log.cc:1196] T 442d90cc310a4627a462c94f969c1782 P 6b51c58c505c44c7943c02c022ddfa18: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387687857264-19106-0/minicluster-data/ts-2/wal/wals/442d90cc310a4627a462c94f969c1782
I20260810 18:48:22.354254 21777 ts_tablet_manager.cc:1939] T 442d90cc310a4627a462c94f969c1782 P 6b51c58c505c44c7943c02c022ddfa18: Deleting consensus metadata
I20260810 18:48:22.354444 21035 catalog_manager.cc:4841] TS 1c06a8266d164fdc98b6891041be38d0 (127.18.168.130:35203): tablet 442d90cc310a4627a462c94f969c1782 (table test-table [id=5b15d56085334d17b8bacd1fbd386587]) successfully deleted
I20260810 18:48:22.355011 21035 catalog_manager.cc:4841] TS 6b51c58c505c44c7943c02c022ddfa18 (127.18.168.131:46491): tablet 442d90cc310a4627a462c94f969c1782 (table test-table [id=5b15d56085334d17b8bacd1fbd386587]) successfully deleted
W20260810 18:48:22.358821 21033 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.168.129:41069 (error 108)
I20260810 18:48:22.358844 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 21289
I20260810 18:48:22.362178 21781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:22.361648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:41069 (local address 127.0.0.1:49428)
0810 18:48:22.361948 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:22.362051 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.129:41069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":154,"threads_started":1}
W20260810 18:48:22.362444 21033 catalog_manager.cc:4584] TS ec02cf09b8914f118a120895e3c0f3cf (127.18.168.129:41069): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 442d90cc310a4627a462c94f969c1782: Network error: Client connection negotiation failed: client connection to 127.18.168.129:41069: connect: Connection refused (error 111)
I20260810 18:48:22.366467 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 21447
I20260810 18:48:22.374197 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 21013
2026-08-10T18:48:22Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6363 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:48:22 dist-test-slave-cbsf krb5kdc[21785](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:48:22 dist-test-slave-cbsf krb5kdc[21785](info): set up 2 sockets
Aug 10 18:48:22 dist-test-slave-cbsf krb5kdc[21785](info): commencing operation
krb5kdc: starting...
W20260810 18:48:23.484047 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:48:23 dist-test-slave-cbsf krb5kdc[21785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:48:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:48:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.168.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
I20260810 18:48:23.607686 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:45073
--webserver_interface=127.18.168.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:34425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:45073
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:23.718407 21799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:23.718719 21799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:23.718791 21799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:23.722747 21799 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:48:23.722837 21799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:23.722878 21799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:23.722935 21799 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:48:23.722973 21799 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:48:23.728350 21799 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:34425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:45073
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:45073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:23.729463 21799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:23.730511 21799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:23.739404 21807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:23.739409 21804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:23.739452 21805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:23.739492 21799 server_base.cc:1034] running on GCE node
I20260810 18:48:23.740288 21799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:23.741417 21799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:23.742583 21799 hybrid_clock.cc:648] HybridClock initialized: now 1786387703742550 us; error 47 us; skew 500 ppm
Aug 10 18:48:23 dist-test-slave-cbsf krb5kdc[21785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387703, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:23.748539 21799 init.cc:377] Logged in from keytab as kudu/127.18.168.190@KRBTEST.COM (short username kudu)
I20260810 18:48:23.749529 21799 webserver.cc:460] Webserver started at http://127.18.168.190:46289/ using document root <none> and password file <none>
I20260810 18:48:23.750113 21799 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:23.750180 21799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:23.750423 21799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:23.752496 21799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "10bee86372ef4e759caa389c8c7ef689"
format_stamp: "Formatted at 2026-08-10 18:48:23 on dist-test-slave-cbsf"
server_key: "59790d89b6e5baec66097b600ad376d9"
server_key_iv: "d1eba31e3bee16e407be681a5f91b35f"
server_key_version: "encryptionkey@0"
I20260810 18:48:23.753171 21799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "10bee86372ef4e759caa389c8c7ef689"
format_stamp: "Formatted at 2026-08-10 18:48:23 on dist-test-slave-cbsf"
server_key: "59790d89b6e5baec66097b600ad376d9"
server_key_iv: "d1eba31e3bee16e407be681a5f91b35f"
server_key_version: "encryptionkey@0"
I20260810 18:48:23.756938 21799 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 18:48:23.759379 21814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:23.760586 21799 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:48:23.760756 21799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "10bee86372ef4e759caa389c8c7ef689"
format_stamp: "Formatted at 2026-08-10 18:48:23 on dist-test-slave-cbsf"
server_key: "59790d89b6e5baec66097b600ad376d9"
server_key_iv: "d1eba31e3bee16e407be681a5f91b35f"
server_key_version: "encryptionkey@0"
I20260810 18:48:23.760869 21799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:23.785497 21799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:23.788511 21799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:23.788748 21799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:23.796096 21799 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:45073
I20260810 18:48:23.796111 21873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:45073 every 8 connection(s)
I20260810 18:48:23.797060 21799 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:48:23.799898 21874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:23.804080 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 21799
I20260810 18:48:23.804219 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:48:23.804498 19106 external_mini_cluster.cc:1428] Setting key 735327a39ccf90c64c23514a20f95cf3
I20260810 18:48:23.805814 21874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10bee86372ef4e759caa389c8c7ef689: Bootstrap starting.
I20260810 18:48:23.808127 21874 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10bee86372ef4e759caa389c8c7ef689: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:23.809041 21874 log.cc:824] T 00000000000000000000000000000000 P 10bee86372ef4e759caa389c8c7ef689: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:23.811179 21874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10bee86372ef4e759caa389c8c7ef689: No bootstrap required, opened a new log
Aug 10 18:48:23 dist-test-slave-cbsf krb5kdc[21785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:23.816022 21874 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10bee86372ef4e759caa389c8c7ef689 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10bee86372ef4e759caa389c8c7ef689" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 45073 } }
I20260810 18:48:23.816298 21874 raft_consensus.cc:374] T 00000000000000000000000000000000 P 10bee86372ef4e759caa389c8c7ef689 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:23.816366 21874 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10bee86372ef4e759caa389c8c7ef689 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10bee86372ef4e759caa389c8c7ef689, State: Initialized, Role: FOLLOWER
I20260810 18:48:23.816907 21874 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10bee86372ef4e759caa389c8c7ef689 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10bee86372ef4e759caa389c8c7ef689" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 45073 } }
I20260810 18:48:23.817106 21874 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10bee86372ef4e759caa389c8c7ef689 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:48:23.817217 21874 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10bee86372ef4e759caa389c8c7ef689 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:48:23.817322 21874 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10bee86372ef4e759caa389c8c7ef689 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:23.818460 21874 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10bee86372ef4e759caa389c8c7ef689 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10bee86372ef4e759caa389c8c7ef689" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 45073 } }
I20260810 18:48:23.818825 21874 leader_election.cc:302] T 00000000000000000000000000000000 P 10bee86372ef4e759caa389c8c7ef689 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10bee86372ef4e759caa389c8c7ef689; no voters:
I20260810 18:48:23.819145 21874 leader_election.cc:288] T 00000000000000000000000000000000 P 10bee86372ef4e759caa389c8c7ef689 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:48:23.819281 21879 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10bee86372ef4e759caa389c8c7ef689 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:23.819543 21879 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10bee86372ef4e759caa389c8c7ef689 [term 1 LEADER]: Becoming Leader. State: Replica: 10bee86372ef4e759caa389c8c7ef689, State: Running, Role: LEADER
I20260810 18:48:23.819844 21877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:23.806037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54340 (local address 127.18.168.190:45073)
0810 18:48:23.806450 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:23.806465 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:23.806499 (+ 34us) server_negotiation.cc:407] Connection header received
0810 18:48:23.807122 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:23.807137 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:23.807440 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:23.807700 (+ 260us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:23.808753 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:23.809926 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:23.811096 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:23.811599 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:23.811713 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:23.814524 (+ 2811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:23.814530 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:23.814549 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:23.814581 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:23.816529 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:23.817076 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:23.817081 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:23.817087 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:23.817284 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:23.817593 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:23.817596 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:23.817598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:23.818103 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:23.818328 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:23.818521 (+ 193us) server_negotiation.cc:299] Negotiation successful
0810 18:48:23.818815 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":135,"threads_started":1}
I20260810 18:48:23.819850 21879 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10bee86372ef4e759caa389c8c7ef689 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10bee86372ef4e759caa389c8c7ef689" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 45073 } }
I20260810 18:48:23.820468 21874 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10bee86372ef4e759caa389c8c7ef689 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:48:23.821748 21882 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10bee86372ef4e759caa389c8c7ef689 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10bee86372ef4e759caa389c8c7ef689. Latest consensus state: current_term: 1 leader_uuid: "10bee86372ef4e759caa389c8c7ef689" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10bee86372ef4e759caa389c8c7ef689" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 45073 } } }
I20260810 18:48:23.821871 21882 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10bee86372ef4e759caa389c8c7ef689 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:23.822307 21881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10bee86372ef4e759caa389c8c7ef689 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "10bee86372ef4e759caa389c8c7ef689" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10bee86372ef4e759caa389c8c7ef689" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 45073 } } }
I20260810 18:48:23.822408 21881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10bee86372ef4e759caa389c8c7ef689 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:23.822324 21889 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:48:23.825502 21889 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:48:23.831023 21889 catalog_manager.cc:1347] Generated new cluster ID: 781817a8a1e34f9695b20bcdf8326a98
I20260810 18:48:23.831094 21889 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:48:23.861135 21889 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:48:23.862435 21889 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:48:23.873080 21889 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 10bee86372ef4e759caa389c8c7ef689: Generated new TSK 0
I20260810 18:48:23.873940 21889 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.168.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab.
Entry for principal kudu/127.18.168.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab.
I20260810 18:48:23.915534 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.129:0
--local_ip_for_outbound_sockets=127.18.168.129
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:45073
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:34425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:24.027915 21902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:24.028183 21902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:24.028251 21902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:24.032194 21902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:24.032289 21902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:24.032385 21902 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.129
I20260810 18:48:24.037549 21902 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:34425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:45073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.129
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:24.038549 21902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:24.039553 21902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:24.049129 21911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:24.049129 21908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:24.049165 21909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:24.049337 21902 server_base.cc:1034] running on GCE node
I20260810 18:48:24.049927 21902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:24.050580 21902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:24.051769 21902 hybrid_clock.cc:648] HybridClock initialized: now 1786387704051756 us; error 25 us; skew 500 ppm
Aug 10 18:48:24 dist-test-slave-cbsf krb5kdc[21785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387704, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:24.055361 21902 init.cc:377] Logged in from keytab as kudu/127.18.168.129@KRBTEST.COM (short username kudu)
I20260810 18:48:24.056420 21902 webserver.cc:460] Webserver started at http://127.18.168.129:42987/ using document root <none> and password file <none>
I20260810 18:48:24.057014 21902 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:24.057075 21902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:24.057276 21902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:24.059454 21902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-0/data/instance:
uuid: "687a7e238cfc4c95bd6fe152461a52e4"
format_stamp: "Formatted at 2026-08-10 18:48:24 on dist-test-slave-cbsf"
server_key: "796a4c75cc6bc76201f4d210318c83c9"
server_key_iv: "db29cc05efd18210c7918c68f171b6a8"
server_key_version: "encryptionkey@0"
I20260810 18:48:24.059973 21902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance:
uuid: "687a7e238cfc4c95bd6fe152461a52e4"
format_stamp: "Formatted at 2026-08-10 18:48:24 on dist-test-slave-cbsf"
server_key: "796a4c75cc6bc76201f4d210318c83c9"
server_key_iv: "db29cc05efd18210c7918c68f171b6a8"
server_key_version: "encryptionkey@0"
I20260810 18:48:24.063691 21902 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260810 18:48:24.066424 21921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:24.067719 21902 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:48:24.067824 21902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-0/wal
uuid: "687a7e238cfc4c95bd6fe152461a52e4"
format_stamp: "Formatted at 2026-08-10 18:48:24 on dist-test-slave-cbsf"
server_key: "796a4c75cc6bc76201f4d210318c83c9"
server_key_iv: "db29cc05efd18210c7918c68f171b6a8"
server_key_version: "encryptionkey@0"
I20260810 18:48:24.067912 21902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:24.085984 21902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:24.088577 21902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:24.088802 21902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:24.089362 21902 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:24.090142 21902 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:24.090191 21902 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:24.090231 21902 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:24.090251 21902 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:24.099514 21902 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.129:33295
I20260810 18:48:24.099575 22045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.129:33295 every 8 connection(s)
I20260810 18:48:24.100448 21902 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
I20260810 18:48:24.102679 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 21902
I20260810 18:48:24.102797 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance
I20260810 18:48:24.103128 19106 external_mini_cluster.cc:1428] Setting key 5340665fe641ed482bdef83a1ba6a9e3
Aug 10 18:48:24 dist-test-slave-cbsf krb5kdc[21785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387704, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.168.130@KRBTEST.COM; defaulting to no policy
I20260810 18:48:24.114900 21877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:24.102317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:49431 (local address 127.18.168.190:45073)
0810 18:48:24.102451 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:24.102455 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:24.103665 (+ 1210us) server_negotiation.cc:407] Connection header received
0810 18:48:24.104829 (+ 1164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:24.104832 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:24.104887 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:24.104962 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:24.106098 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.106802 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.107957 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.108372 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.108497 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:24.111016 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:24.111020 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:24.111025 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:24.111062 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:24.112905 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:24.113385 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:24.113391 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:24.113393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:24.113486 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:24.113818 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:24.113821 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:24.113823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:24.114081 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:24.114195 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:24.114674 (+ 479us) server_negotiation.cc:299] Negotiation successful
0810 18:48:24.114779 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 18:48:24.115778 22048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:24.102735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:45073 (local address 127.18.168.129:49431)
0810 18:48:24.103458 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:24.103531 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:24.104566 (+ 1035us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:24.105126 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:24.105132 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:24.105489 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:24.105934 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:24.105946 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.106938 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.106941 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:24.107826 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:24.107835 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.108517 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.108520 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:24.108665 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:24.109315 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:24.109332 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:24.110833 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:24.113020 (+ 2187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:24.113026 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:24.113039 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:24.113276 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:24.113592 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:24.113594 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:24.113596 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:24.113727 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:24.114198 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:24.114204 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:24.114441 (+ 237us) client_negotiation.cc:769] Sending connection context
0810 18:48:24.114606 (+ 165us) client_negotiation.cc:240] Negotiation successful
0810 18:48:24.114839 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":134,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.130@KRBTEST.COM" created.
I20260810 18:48:24.117030 22046 heartbeater.cc:347] Connected to a master server at 127.18.168.190:45073
I20260810 18:48:24.117282 22046 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:24.117782 22046 heartbeater.cc:511] Master 127.18.168.190:45073 requested a full tablet report, sending...
I20260810 18:48:24.119305 21835 ts_manager.cc:194] Registered new tserver with Master: 687a7e238cfc4c95bd6fe152461a52e4 (127.18.168.129:33295)
I20260810 18:48:24.120508 21835 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.129@KRBTEST.COM'} at 127.18.168.129:49431
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab.
Entry for principal kudu/127.18.168.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab.
I20260810 18:48:24.134018 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.130:0
--local_ip_for_outbound_sockets=127.18.168.130
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:45073
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:34425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:24.242559 22052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:24.242786 22052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:24.242826 22052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:24.246841 22052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:24.246889 22052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:24.246987 22052 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.130
I20260810 18:48:24.252110 22052 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:34425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:45073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.130
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:24.253113 22052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:24.254047 22052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:24.263621 22062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:24.263676 22058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:24.263610 22060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:24.263901 22052 server_base.cc:1034] running on GCE node
I20260810 18:48:24.264293 22052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:24.264994 22052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:24.266292 22052 hybrid_clock.cc:648] HybridClock initialized: now 1786387704266275 us; error 25 us; skew 500 ppm
Aug 10 18:48:24 dist-test-slave-cbsf krb5kdc[21785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387704, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:24.268975 22052 init.cc:377] Logged in from keytab as kudu/127.18.168.130@KRBTEST.COM (short username kudu)
I20260810 18:48:24.269869 22052 webserver.cc:460] Webserver started at http://127.18.168.130:42379/ using document root <none> and password file <none>
I20260810 18:48:24.270433 22052 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:24.270524 22052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:24.270787 22052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:24.272902 22052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-1/data/instance:
uuid: "ebaa9297840b4e918f551b706f3744d2"
format_stamp: "Formatted at 2026-08-10 18:48:24 on dist-test-slave-cbsf"
server_key: "73abacf42da38bc907eb2afa6ed634b0"
server_key_iv: "b9ac41f4b3f3bb043fc505ce5f41869a"
server_key_version: "encryptionkey@0"
I20260810 18:48:24.273465 22052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance:
uuid: "ebaa9297840b4e918f551b706f3744d2"
format_stamp: "Formatted at 2026-08-10 18:48:24 on dist-test-slave-cbsf"
server_key: "73abacf42da38bc907eb2afa6ed634b0"
server_key_iv: "b9ac41f4b3f3bb043fc505ce5f41869a"
server_key_version: "encryptionkey@0"
I20260810 18:48:24.276861 22052 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:48:24.279194 22071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:24.280418 22052 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:48:24.280562 22052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-1/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-1/wal
uuid: "ebaa9297840b4e918f551b706f3744d2"
format_stamp: "Formatted at 2026-08-10 18:48:24 on dist-test-slave-cbsf"
server_key: "73abacf42da38bc907eb2afa6ed634b0"
server_key_iv: "b9ac41f4b3f3bb043fc505ce5f41869a"
server_key_version: "encryptionkey@0"
I20260810 18:48:24.280669 22052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:24.294721 22052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:24.297387 22052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:24.297591 22052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:24.298156 22052 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:24.298990 22052 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:24.299064 22052 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:24.299132 22052 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:24.299180 22052 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:24.307818 22052 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.130:37893
I20260810 18:48:24.307840 22197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.130:37893 every 8 connection(s)
I20260810 18:48:24.308655 22052 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
I20260810 18:48:24.310249 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 22052
I20260810 18:48:24.310398 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance
I20260810 18:48:24.310755 19106 external_mini_cluster.cc:1428] Setting key 598186de0789a1e32dc100d044fc1e9a
Aug 10 18:48:24 dist-test-slave-cbsf krb5kdc[21785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387704, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.168.131@KRBTEST.COM; defaulting to no policy
I20260810 18:48:24.323658 21877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:24.310494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:56737 (local address 127.18.168.190:45073)
0810 18:48:24.310657 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:24.310662 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:24.311750 (+ 1088us) server_negotiation.cc:407] Connection header received
0810 18:48:24.312630 (+ 880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:24.312636 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:24.312714 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:24.312801 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:24.313895 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.314599 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.316324 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.316744 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.316862 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:24.319791 (+ 2929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:24.319795 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:24.319797 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:24.319818 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:24.321452 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:24.322055 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:24.322058 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:24.322060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:24.322161 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:24.322528 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:24.322532 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:24.322534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:24.322824 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:24.322918 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:24.323411 (+ 493us) server_negotiation.cc:299] Negotiation successful
0810 18:48:24.323529 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 18:48:24.324589 22200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:24.310920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:45073 (local address 127.18.168.130:56737)
0810 18:48:24.311608 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:24.311659 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:24.312426 (+ 767us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:24.312929 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:24.312935 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:24.313263 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:24.313745 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:24.313758 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.314749 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.314755 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:24.316190 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:24.316201 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.316886 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.316890 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:24.317053 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:24.317716 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:24.317735 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:24.319613 (+ 1878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:24.321587 (+ 1974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:24.321594 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:24.321615 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:24.321929 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:24.322296 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:24.322299 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:24.322301 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:24.322427 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:24.322931 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:24.322936 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:24.323160 (+ 224us) client_negotiation.cc:769] Sending connection context
0810 18:48:24.323355 (+ 195us) client_negotiation.cc:240] Negotiation successful
0810 18:48:24.323656 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":129,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.131@KRBTEST.COM" created.
I20260810 18:48:24.325740 22198 heartbeater.cc:347] Connected to a master server at 127.18.168.190:45073
I20260810 18:48:24.325987 22198 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:24.326445 22198 heartbeater.cc:511] Master 127.18.168.190:45073 requested a full tablet report, sending...
I20260810 18:48:24.327657 21835 ts_manager.cc:194] Registered new tserver with Master: ebaa9297840b4e918f551b706f3744d2 (127.18.168.130:37893)
I20260810 18:48:24.328220 21835 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.130@KRBTEST.COM'} at 127.18.168.130:56737
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab.
Entry for principal kudu/127.18.168.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab.
I20260810 18:48:24.343277 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.131:0
--local_ip_for_outbound_sockets=127.18.168.131
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:45073
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:34425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:24.452870 22203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:24.453086 22203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:24.453127 22203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:24.456974 22203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:24.457018 22203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:24.457069 22203 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.131
I20260810 18:48:24.462083 22203 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:34425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:45073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22203
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.131
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:24.463100 22203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:24.464293 22203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:24.473935 22210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:24.473979 22212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:24.473943 22209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:24.474169 22203 server_base.cc:1034] running on GCE node
I20260810 18:48:24.474714 22203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:24.475363 22203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:24.476550 22203 hybrid_clock.cc:648] HybridClock initialized: now 1786387704476540 us; error 28 us; skew 500 ppm
Aug 10 18:48:24 dist-test-slave-cbsf krb5kdc[21785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387704, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:24.479274 22203 init.cc:377] Logged in from keytab as kudu/127.18.168.131@KRBTEST.COM (short username kudu)
I20260810 18:48:24.480134 22203 webserver.cc:460] Webserver started at http://127.18.168.131:41245/ using document root <none> and password file <none>
I20260810 18:48:24.480736 22203 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:24.480835 22203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:24.481065 22203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:24.483019 22203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-2/data/instance:
uuid: "ed88de8528a245f7875bed1c3009eaaa"
format_stamp: "Formatted at 2026-08-10 18:48:24 on dist-test-slave-cbsf"
server_key: "03bb6bdb9068a7ac0437879597767da4"
server_key_iv: "ad72511503b70e15146e3e51adb5bfa7"
server_key_version: "encryptionkey@0"
I20260810 18:48:24.483608 22203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance:
uuid: "ed88de8528a245f7875bed1c3009eaaa"
format_stamp: "Formatted at 2026-08-10 18:48:24 on dist-test-slave-cbsf"
server_key: "03bb6bdb9068a7ac0437879597767da4"
server_key_iv: "ad72511503b70e15146e3e51adb5bfa7"
server_key_version: "encryptionkey@0"
I20260810 18:48:24.487529 22203 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:48:24.489961 22220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:24.491132 22203 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:48:24.491293 22203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-2/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-2/wal
uuid: "ed88de8528a245f7875bed1c3009eaaa"
format_stamp: "Formatted at 2026-08-10 18:48:24 on dist-test-slave-cbsf"
server_key: "03bb6bdb9068a7ac0437879597767da4"
server_key_iv: "ad72511503b70e15146e3e51adb5bfa7"
server_key_version: "encryptionkey@0"
I20260810 18:48:24.491400 22203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:24.505934 22203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:24.508961 22203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:24.509155 22203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:24.509760 22203 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:24.510602 22203 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:24.510671 22203 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:24.510735 22203 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:24.510778 22203 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:24.519394 22203 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.131:36231
I20260810 18:48:24.519414 22344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.131:36231 every 8 connection(s)
I20260810 18:48:24.520296 22203 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
Aug 10 18:48:24 dist-test-slave-cbsf krb5kdc[21785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387704, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:24.529314 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 22203
I20260810 18:48:24.529431 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance
I20260810 18:48:24.529724 19106 external_mini_cluster.cc:1428] Setting key 299141f1ba428d862e1dadbfbd5c578e
I20260810 18:48:24.533764 21877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:24.522208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:46143 (local address 127.18.168.190:45073)
0810 18:48:24.522362 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:24.522366 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:24.523323 (+ 957us) server_negotiation.cc:407] Connection header received
0810 18:48:24.524115 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:24.524117 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:24.524161 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:24.524229 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:24.525328 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.525935 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.527135 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.527382 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.527497 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:24.529986 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:24.529992 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:24.529999 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:24.530023 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:24.531802 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:24.532391 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:24.532394 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:24.532395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:24.532492 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:24.532865 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:24.532867 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:24.532868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:24.533038 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:24.533116 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:24.533557 (+ 441us) server_negotiation.cc:299] Negotiation successful
0810 18:48:24.533652 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 18:48:24.534564 22348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:24.522563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:45073 (local address 127.18.168.131:46143)
0810 18:48:24.523175 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:24.523212 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:24.523943 (+ 731us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:24.524344 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:24.524350 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:24.524689 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:24.525210 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:24.525222 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.526081 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.526084 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:24.526979 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:24.526989 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.527552 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.527557 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:24.527741 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:24.528235 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:24.528251 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:24.529811 (+ 1560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:24.531929 (+ 2118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:24.531936 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:24.531949 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:24.532295 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:24.532582 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:24.532584 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:24.532585 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:24.532771 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:24.533126 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:24.533131 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:24.533351 (+ 220us) client_negotiation.cc:769] Sending connection context
0810 18:48:24.533493 (+ 142us) client_negotiation.cc:240] Negotiation successful
0810 18:48:24.533732 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":109,"threads_started":1}
I20260810 18:48:24.535671 22345 heartbeater.cc:347] Connected to a master server at 127.18.168.190:45073
I20260810 18:48:24.535904 22345 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:24.536466 22345 heartbeater.cc:511] Master 127.18.168.190:45073 requested a full tablet report, sending...
I20260810 18:48:24.537520 21835 ts_manager.cc:194] Registered new tserver with Master: ed88de8528a245f7875bed1c3009eaaa (127.18.168.131:36231)
I20260810 18:48:24.538110 21835 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.131@KRBTEST.COM'} at 127.18.168.131:46143
I20260810 18:48:24.543833 19106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 18:48:24.558804 21877 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:24.557414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54342 (local address 127.18.168.190:45073)
0810 18:48:24.557560 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:24.557565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:24.557751 (+ 186us) server_negotiation.cc:407] Connection header received
0810 18:48:24.557857 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:24.557859 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:24.557901 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:24.557967 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:24.558675 (+ 708us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54342: BlockingRecv error: recv got EOF from 127.0.0.1:54342 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 18:48:24.559801 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 21902
I20260810 18:48:24.570662 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 22052
I20260810 18:48:24.584129 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 22203
I20260810 18:48:24.592443 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 21799
2026-08-10T18:48:24Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2214 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:48:24 dist-test-slave-cbsf krb5kdc[22358](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:48:24 dist-test-slave-cbsf krb5kdc[22358](info): set up 2 sockets
Aug 10 18:48:24 dist-test-slave-cbsf krb5kdc[22358](info): commencing operation
krb5kdc: starting...
W20260810 18:48:26.176554 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.562s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:48:26 dist-test-slave-cbsf krb5kdc[22358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387706, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:48:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:48:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.168.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
I20260810 18:48:26.361097 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:37951
--webserver_interface=127.18.168.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:37899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:37951
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:26.554965 22375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:26.555490 22375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:26.555694 22375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:26.562137 22375 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:48:26.562251 22375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:26.562362 22375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:26.562440 22375 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:48:26.562521 22375 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:48:26.570878 22375 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:37899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:37951
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:37951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:26.572752 22375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:26.574590 22375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:26.588438 22383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:26.588372 22380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:26.588397 22381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:26.589402 22375 server_base.cc:1034] running on GCE node
I20260810 18:48:26.590452 22375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:26.592407 22375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:26.593659 22375 hybrid_clock.cc:648] HybridClock initialized: now 1786387706593633 us; error 38 us; skew 500 ppm
Aug 10 18:48:26 dist-test-slave-cbsf krb5kdc[22358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387706, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:26.598156 22375 init.cc:377] Logged in from keytab as kudu/127.18.168.190@KRBTEST.COM (short username kudu)
I20260810 18:48:26.599684 22375 webserver.cc:460] Webserver started at http://127.18.168.190:45705/ using document root <none> and password file <none>
I20260810 18:48:26.600777 22375 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:26.600929 22375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:26.601346 22375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:26.605072 22375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "7473f5d948e943dca9e41ee833100543"
format_stamp: "Formatted at 2026-08-10 18:48:26 on dist-test-slave-cbsf"
server_key: "67b7a1ecf199820d9098fede2f545f7c"
server_key_iv: "af51eb2da8770953bb2b578d9cb4bdd9"
server_key_version: "encryptionkey@0"
I20260810 18:48:26.606074 22375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "7473f5d948e943dca9e41ee833100543"
format_stamp: "Formatted at 2026-08-10 18:48:26 on dist-test-slave-cbsf"
server_key: "67b7a1ecf199820d9098fede2f545f7c"
server_key_iv: "af51eb2da8770953bb2b578d9cb4bdd9"
server_key_version: "encryptionkey@0"
I20260810 18:48:26.612821 22375 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260810 18:48:26.617069 22390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:26.619004 22375 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260810 18:48:26.619264 22375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "7473f5d948e943dca9e41ee833100543"
format_stamp: "Formatted at 2026-08-10 18:48:26 on dist-test-slave-cbsf"
server_key: "67b7a1ecf199820d9098fede2f545f7c"
server_key_iv: "af51eb2da8770953bb2b578d9cb4bdd9"
server_key_version: "encryptionkey@0"
I20260810 18:48:26.619426 22375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:26.635129 22375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:26.640050 22375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:26.640359 22375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:26.653191 22375 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:37951
I20260810 18:48:26.653741 22451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:37951 every 8 connection(s)
I20260810 18:48:26.654870 22375 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:48:26.660283 22452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:26.661787 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 22375
I20260810 18:48:26.661924 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:48:26.662451 19106 external_mini_cluster.cc:1428] Setting key 4d9d8bc6dbb3a827bab2d4f4057e7556
I20260810 18:48:26.674186 22452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7473f5d948e943dca9e41ee833100543: Bootstrap starting.
I20260810 18:48:26.679392 22452 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7473f5d948e943dca9e41ee833100543: Neither blocks nor log segments found. Creating new log.
Aug 10 18:48:26 dist-test-slave-cbsf krb5kdc[22358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387706, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:26.681200 22452 log.cc:824] T 00000000000000000000000000000000 P 7473f5d948e943dca9e41ee833100543: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:26.686329 22452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7473f5d948e943dca9e41ee833100543: No bootstrap required, opened a new log
I20260810 18:48:26.692368 22456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:26.664831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59016 (local address 127.18.168.190:37951)
0810 18:48:26.665595 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:26.665631 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:26.665695 (+ 64us) server_negotiation.cc:407] Connection header received
0810 18:48:26.667391 (+ 1696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:26.667427 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:26.668263 (+ 836us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:26.669096 (+ 833us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:26.670211 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:26.673395 (+ 3184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:26.674813 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:26.676501 (+ 1688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:26.676972 (+ 471us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:26.681488 (+ 4516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:26.681509 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:26.681553 (+ 44us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:26.681617 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:26.685894 (+ 4277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:26.686805 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:26.686815 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:26.686852 (+ 37us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:26.687221 (+ 369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:26.687629 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:26.687644 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:26.687646 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:26.688674 (+ 1028us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:26.689054 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:26.689387 (+ 333us) server_negotiation.cc:299] Negotiation successful
0810 18:48:26.689993 (+ 606us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":189,"threads_started":1}
I20260810 18:48:26.695034 22452 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7473f5d948e943dca9e41ee833100543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7473f5d948e943dca9e41ee833100543" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 37951 } }
I20260810 18:48:26.695538 22452 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7473f5d948e943dca9e41ee833100543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:26.695653 22452 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7473f5d948e943dca9e41ee833100543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7473f5d948e943dca9e41ee833100543, State: Initialized, Role: FOLLOWER
I20260810 18:48:26.696457 22452 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7473f5d948e943dca9e41ee833100543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7473f5d948e943dca9e41ee833100543" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 37951 } }
I20260810 18:48:26.696741 22452 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7473f5d948e943dca9e41ee833100543 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:48:26.696892 22452 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7473f5d948e943dca9e41ee833100543 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:48:26.697201 19106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:48:26.697139 22452 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7473f5d948e943dca9e41ee833100543 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:26.699790 22452 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7473f5d948e943dca9e41ee833100543 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7473f5d948e943dca9e41ee833100543" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 37951 } }
I20260810 18:48:26.700611 22452 leader_election.cc:302] T 00000000000000000000000000000000 P 7473f5d948e943dca9e41ee833100543 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7473f5d948e943dca9e41ee833100543; no voters:
I20260810 18:48:26.701339 22452 leader_election.cc:288] T 00000000000000000000000000000000 P 7473f5d948e943dca9e41ee833100543 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:48:26.701479 22458 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7473f5d948e943dca9e41ee833100543 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:26.702071 22458 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7473f5d948e943dca9e41ee833100543 [term 1 LEADER]: Becoming Leader. State: Replica: 7473f5d948e943dca9e41ee833100543, State: Running, Role: LEADER
I20260810 18:48:26.702600 22458 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7473f5d948e943dca9e41ee833100543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7473f5d948e943dca9e41ee833100543" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 37951 } }
I20260810 18:48:26.703697 22452 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7473f5d948e943dca9e41ee833100543 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:48:26.704823 22458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7473f5d948e943dca9e41ee833100543 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7473f5d948e943dca9e41ee833100543. Latest consensus state: current_term: 1 leader_uuid: "7473f5d948e943dca9e41ee833100543" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7473f5d948e943dca9e41ee833100543" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 37951 } } }
I20260810 18:48:26.705036 22458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7473f5d948e943dca9e41ee833100543 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:26.705420 22458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7473f5d948e943dca9e41ee833100543 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7473f5d948e943dca9e41ee833100543" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7473f5d948e943dca9e41ee833100543" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 37951 } } }
I20260810 18:48:26.705590 22458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7473f5d948e943dca9e41ee833100543 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:26.706076 22467 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:48:26.713232 22467 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:48:26.724534 22467 catalog_manager.cc:1347] Generated new cluster ID: de44cbc5a26f4b21883988ee5327cd68
I20260810 18:48:26.724640 22467 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:48:26.738286 22467 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:48:26.739427 22456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:26.728002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59018 (local address 127.18.168.190:37951)
0810 18:48:26.728223 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:26.728230 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:26.728616 (+ 386us) server_negotiation.cc:407] Connection header received
0810 18:48:26.729003 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:26.729007 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:26.729094 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:26.729227 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:26.730669 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:26.732017 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:26.734660 (+ 2643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:26.735301 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:26.735537 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:26.737427 (+ 1890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:26.737433 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:26.737451 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:48:26.738478 (+ 1027us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:26.738542 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:48:26.738586 (+ 44us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:26.738791 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:26.739086 (+ 295us) server_negotiation.cc:299] Negotiation successful
0810 18:48:26.739194 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 18:48:26.739904 22467 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:48:26.751890 22467 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7473f5d948e943dca9e41ee833100543: Generated new TSK 0
I20260810 18:48:26.753469 22467 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:48:26.767515 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 22375
2026-08-10T18:48:26Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2183 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:48:26 dist-test-slave-cbsf krb5kdc[22491](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:48:26 dist-test-slave-cbsf krb5kdc[22491](info): set up 2 sockets
Aug 10 18:48:26 dist-test-slave-cbsf krb5kdc[22491](info): commencing operation
krb5kdc: starting...
W20260810 18:48:28.349210 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.549s user 0.000s sys 0.013s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:48:28 dist-test-slave-cbsf krb5kdc[22491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:48:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:48:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.168.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
I20260810 18:48:28.532786 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:45849
--webserver_interface=127.18.168.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:34587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:45849
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:28.758114 22504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:28.758611 22504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:28.758771 22504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:28.765484 22504 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:48:28.765599 22504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:28.765705 22504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:28.765799 22504 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:48:28.765901 22504 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:48:28.774554 22504 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:34587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:45849
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:45849
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:28.776552 22504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:28.778498 22504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:28.793629 22510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:28.793637 22511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:28.793637 22514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:28.794230 22504 server_base.cc:1034] running on GCE node
I20260810 18:48:28.795373 22504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:28.797329 22504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:28.799031 22504 hybrid_clock.cc:648] HybridClock initialized: now 1786387708798965 us; error 74 us; skew 500 ppm
Aug 10 18:48:28 dist-test-slave-cbsf krb5kdc[22491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387708, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:28.809178 22504 init.cc:377] Logged in from keytab as kudu/127.18.168.190@KRBTEST.COM (short username kudu)
I20260810 18:48:28.810988 22504 webserver.cc:460] Webserver started at http://127.18.168.190:39883/ using document root <none> and password file <none>
I20260810 18:48:28.811975 22504 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:28.812091 22504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:28.812405 22504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:28.816648 22504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "78ea1cc2e9d5490eaa573f8cfdfc8e8c"
format_stamp: "Formatted at 2026-08-10 18:48:28 on dist-test-slave-cbsf"
server_key: "65c59cbb0371e753fb82d61c06ff56c3"
server_key_iv: "dc8f0424ffc88b2eec116e3b7b1a1202"
server_key_version: "encryptionkey@0"
I20260810 18:48:28.817870 22504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "78ea1cc2e9d5490eaa573f8cfdfc8e8c"
format_stamp: "Formatted at 2026-08-10 18:48:28 on dist-test-slave-cbsf"
server_key: "65c59cbb0371e753fb82d61c06ff56c3"
server_key_iv: "dc8f0424ffc88b2eec116e3b7b1a1202"
server_key_version: "encryptionkey@0"
I20260810 18:48:28.825070 22504 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260810 18:48:28.830353 22522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:28.832593 22504 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:48:28.832799 22504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "78ea1cc2e9d5490eaa573f8cfdfc8e8c"
format_stamp: "Formatted at 2026-08-10 18:48:28 on dist-test-slave-cbsf"
server_key: "65c59cbb0371e753fb82d61c06ff56c3"
server_key_iv: "dc8f0424ffc88b2eec116e3b7b1a1202"
server_key_version: "encryptionkey@0"
I20260810 18:48:28.832942 22504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:28.873678 22504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:28.877916 22504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:28.878082 22504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:28.885234 22504 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:45849
I20260810 18:48:28.885533 22579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:45849 every 8 connection(s)
I20260810 18:48:28.886245 22504 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:48:28.889075 22580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:28.894507 22580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78ea1cc2e9d5490eaa573f8cfdfc8e8c: Bootstrap starting.
I20260810 18:48:28.896299 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 22504
I20260810 18:48:28.896406 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:48:28.896778 19106 external_mini_cluster.cc:1428] Setting key 4fefb691295bcd79d1a8fc362cd57ce9
I20260810 18:48:28.897182 22580 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 78ea1cc2e9d5490eaa573f8cfdfc8e8c: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:28.898056 22580 log.cc:824] T 00000000000000000000000000000000 P 78ea1cc2e9d5490eaa573f8cfdfc8e8c: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:28.900256 22580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78ea1cc2e9d5490eaa573f8cfdfc8e8c: No bootstrap required, opened a new log
I20260810 18:48:28.904206 22580 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78ea1cc2e9d5490eaa573f8cfdfc8e8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78ea1cc2e9d5490eaa573f8cfdfc8e8c" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 45849 } }
I20260810 18:48:28.904507 22580 raft_consensus.cc:374] T 00000000000000000000000000000000 P 78ea1cc2e9d5490eaa573f8cfdfc8e8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:28.904583 22580 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78ea1cc2e9d5490eaa573f8cfdfc8e8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78ea1cc2e9d5490eaa573f8cfdfc8e8c, State: Initialized, Role: FOLLOWER
I20260810 18:48:28.905027 22580 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78ea1cc2e9d5490eaa573f8cfdfc8e8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78ea1cc2e9d5490eaa573f8cfdfc8e8c" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 45849 } }
I20260810 18:48:28.905150 22580 raft_consensus.cc:388] T 00000000000000000000000000000000 P 78ea1cc2e9d5490eaa573f8cfdfc8e8c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:48:28.905196 22580 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78ea1cc2e9d5490eaa573f8cfdfc8e8c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:48:28.905267 22580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78ea1cc2e9d5490eaa573f8cfdfc8e8c [term 0 FOLLOWER]: Advancing to term 1
Aug 10 18:48:28 dist-test-slave-cbsf krb5kdc[22491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:28.906448 22580 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78ea1cc2e9d5490eaa573f8cfdfc8e8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78ea1cc2e9d5490eaa573f8cfdfc8e8c" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 45849 } }
I20260810 18:48:28.906809 22580 leader_election.cc:302] T 00000000000000000000000000000000 P 78ea1cc2e9d5490eaa573f8cfdfc8e8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 78ea1cc2e9d5490eaa573f8cfdfc8e8c; no voters:
I20260810 18:48:28.907133 22580 leader_election.cc:288] T 00000000000000000000000000000000 P 78ea1cc2e9d5490eaa573f8cfdfc8e8c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:48:28.907353 22586 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78ea1cc2e9d5490eaa573f8cfdfc8e8c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:28.907672 22586 raft_consensus.cc:686] T 00000000000000000000000000000000 P 78ea1cc2e9d5490eaa573f8cfdfc8e8c [term 1 LEADER]: Becoming Leader. State: Replica: 78ea1cc2e9d5490eaa573f8cfdfc8e8c, State: Running, Role: LEADER
I20260810 18:48:28.907967 22586 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78ea1cc2e9d5490eaa573f8cfdfc8e8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78ea1cc2e9d5490eaa573f8cfdfc8e8c" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 45849 } }
I20260810 18:48:28.908464 22580 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78ea1cc2e9d5490eaa573f8cfdfc8e8c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:48:28.909902 22588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78ea1cc2e9d5490eaa573f8cfdfc8e8c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78ea1cc2e9d5490eaa573f8cfdfc8e8c. Latest consensus state: current_term: 1 leader_uuid: "78ea1cc2e9d5490eaa573f8cfdfc8e8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78ea1cc2e9d5490eaa573f8cfdfc8e8c" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 45849 } } }
I20260810 18:48:28.909909 22587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78ea1cc2e9d5490eaa573f8cfdfc8e8c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "78ea1cc2e9d5490eaa573f8cfdfc8e8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78ea1cc2e9d5490eaa573f8cfdfc8e8c" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 45849 } } }
I20260810 18:48:28.910039 22588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78ea1cc2e9d5490eaa573f8cfdfc8e8c [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:28.910074 22587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78ea1cc2e9d5490eaa573f8cfdfc8e8c [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:28.910754 22598 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:48:28.912736 22584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:28.897996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43390 (local address 127.18.168.190:45849)
0810 18:48:28.898473 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:28.898488 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:28.898524 (+ 36us) server_negotiation.cc:407] Connection header received
0810 18:48:28.899349 (+ 825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:28.899374 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:28.899735 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:28.900070 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:28.901027 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:28.902028 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:28.902927 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:28.903412 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:28.903593 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:28.906739 (+ 3146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:28.906744 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:28.906757 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:28.906784 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:28.909093 (+ 2309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:28.909645 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:28.909664 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:28.909673 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:28.909828 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:28.910173 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:28.910177 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:28.910181 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:28.910718 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:28.910919 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:28.911714 (+ 795us) server_negotiation.cc:299] Negotiation successful
0810 18:48:28.911991 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":154,"threads_started":1}
I20260810 18:48:28.914111 22598 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:48:28.919819 22598 catalog_manager.cc:1347] Generated new cluster ID: 5ec21a715a33464e8882e8c932271ebd
I20260810 18:48:28.919886 22598 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:48:28.935055 22598 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:48:28.935987 22598 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:48:28.939996 22598 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 78ea1cc2e9d5490eaa573f8cfdfc8e8c: Generated new TSK 0
I20260810 18:48:28.940677 22598 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.168.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab.
Entry for principal kudu/127.18.168.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab.
I20260810 18:48:28.976855 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.129:0
--local_ip_for_outbound_sockets=127.18.168.129
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:45849
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:34587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:29.111184 22612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:29.111455 22612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:29.111523 22612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:29.116225 22612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:29.116310 22612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:29.116396 22612 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.129
I20260810 18:48:29.121733 22612 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:34587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:45849
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.129
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:29.122807 22612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:29.123967 22612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:29.133693 22621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:29.133671 22617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:29.133668 22618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:29.133807 22612 server_base.cc:1034] running on GCE node
I20260810 18:48:29.134364 22612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:29.135023 22612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:29.136216 22612 hybrid_clock.cc:648] HybridClock initialized: now 1786387709136184 us; error 50 us; skew 500 ppm
Aug 10 18:48:29 dist-test-slave-cbsf krb5kdc[22491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387709, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:29.139150 22612 init.cc:377] Logged in from keytab as kudu/127.18.168.129@KRBTEST.COM (short username kudu)
I20260810 18:48:29.140066 22612 webserver.cc:460] Webserver started at http://127.18.168.129:46513/ using document root <none> and password file <none>
I20260810 18:48:29.140764 22612 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:29.140843 22612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:29.141014 22612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:29.142988 22612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-0/data/instance:
uuid: "9113fad922854f6c86aff72c45b66dc6"
format_stamp: "Formatted at 2026-08-10 18:48:29 on dist-test-slave-cbsf"
server_key: "d11176e99a1e1064ee4fe495e54e6989"
server_key_iv: "1bb899d0453b76ad2d012bbe7d49f430"
server_key_version: "encryptionkey@0"
I20260810 18:48:29.143496 22612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance:
uuid: "9113fad922854f6c86aff72c45b66dc6"
format_stamp: "Formatted at 2026-08-10 18:48:29 on dist-test-slave-cbsf"
server_key: "d11176e99a1e1064ee4fe495e54e6989"
server_key_iv: "1bb899d0453b76ad2d012bbe7d49f430"
server_key_version: "encryptionkey@0"
I20260810 18:48:29.147223 22612 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 18:48:29.149560 22628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:29.150781 22612 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:48:29.150918 22612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-0/wal
uuid: "9113fad922854f6c86aff72c45b66dc6"
format_stamp: "Formatted at 2026-08-10 18:48:29 on dist-test-slave-cbsf"
server_key: "d11176e99a1e1064ee4fe495e54e6989"
server_key_iv: "1bb899d0453b76ad2d012bbe7d49f430"
server_key_version: "encryptionkey@0"
I20260810 18:48:29.151015 22612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:29.172042 22612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:29.174829 22612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:29.175033 22612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:29.175635 22612 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:29.176456 22612 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:29.176503 22612 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:29.176582 22612 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:29.176613 22612 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:29.185439 22612 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.129:36099
I20260810 18:48:29.185472 22758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.129:36099 every 8 connection(s)
I20260810 18:48:29.186311 22612 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
I20260810 18:48:29.195178 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 22612
I20260810 18:48:29.195320 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance
I20260810 18:48:29.195734 19106 external_mini_cluster.cc:1428] Setting key fb3b5cc3b0343a4ec465cebfcf6443a3
Aug 10 18:48:29 dist-test-slave-cbsf krb5kdc[22491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387709, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:29.201092 22584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:29.188167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:50059 (local address 127.18.168.190:45849)
0810 18:48:29.188309 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:29.188314 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:29.189267 (+ 953us) server_negotiation.cc:407] Connection header received
0810 18:48:29.190664 (+ 1397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:29.190667 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:29.190729 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:29.190835 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:29.191995 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.192541 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.193748 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.194014 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.194128 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:29.196975 (+ 2847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:29.196978 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:29.196980 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:29.197005 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:29.198655 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:29.199337 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:29.199342 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:29.199345 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:29.199441 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:29.199910 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:29.199913 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:29.199917 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:29.200112 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:29.200190 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:29.200863 (+ 673us) server_negotiation.cc:299] Negotiation successful
0810 18:48:29.200967 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 18:48:29.201984 22761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:29.188590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:45849 (local address 127.18.168.129:50059)
0810 18:48:29.189100 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:29.189142 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:29.190435 (+ 1293us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:29.191014 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:29.191021 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:29.191378 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:29.191837 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:29.191849 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.192687 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.192692 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:29.193615 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:29.193625 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.194178 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.194183 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:29.194366 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:29.194947 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:29.194971 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:29.196801 (+ 1830us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:29.198784 (+ 1983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:29.198793 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:29.198811 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:29.199207 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:29.199570 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:29.199577 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:29.199581 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:29.199787 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:29.200224 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:29.200231 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:29.200582 (+ 351us) client_negotiation.cc:769] Sending connection context
0810 18:48:29.200819 (+ 237us) client_negotiation.cc:240] Negotiation successful
0810 18:48:29.201195 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":129,"threads_started":1}
I20260810 18:48:29.203351 22759 heartbeater.cc:347] Connected to a master server at 127.18.168.190:45849
I20260810 18:48:29.203588 22759 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:29.204080 22759 heartbeater.cc:511] Master 127.18.168.190:45849 requested a full tablet report, sending...
I20260810 18:48:29.205694 22541 ts_manager.cc:194] Registered new tserver with Master: 9113fad922854f6c86aff72c45b66dc6 (127.18.168.129:36099)
WARNING: no policy specified for kudu/127.18.168.130@KRBTEST.COM; defaulting to no policy
I20260810 18:48:29.207046 22541 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.129@KRBTEST.COM'} at 127.18.168.129:50059
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab.
Entry for principal kudu/127.18.168.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab.
I20260810 18:48:29.227449 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.130:0
--local_ip_for_outbound_sockets=127.18.168.130
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:45849
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:34587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:29.340422 22764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:29.340808 22764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:29.340917 22764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:29.347784 22764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:29.347873 22764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:29.347960 22764 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.130
I20260810 18:48:29.356894 22764 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:34587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:45849
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.130
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:29.358323 22764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:29.359812 22764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:29.372018 22771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:29.372026 22773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:29.372025 22769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:29.372454 22764 server_base.cc:1034] running on GCE node
I20260810 18:48:29.372931 22764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:29.373687 22764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:29.374879 22764 hybrid_clock.cc:648] HybridClock initialized: now 1786387709374846 us; error 47 us; skew 500 ppm
Aug 10 18:48:29 dist-test-slave-cbsf krb5kdc[22491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387709, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:29.377745 22764 init.cc:377] Logged in from keytab as kudu/127.18.168.130@KRBTEST.COM (short username kudu)
I20260810 18:48:29.378623 22764 webserver.cc:460] Webserver started at http://127.18.168.130:37191/ using document root <none> and password file <none>
I20260810 18:48:29.379176 22764 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:29.379237 22764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:29.379448 22764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:29.381524 22764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-1/data/instance:
uuid: "bd524e1076f2407887e86e92fcb7f717"
format_stamp: "Formatted at 2026-08-10 18:48:29 on dist-test-slave-cbsf"
server_key: "30c413aadee57f0c51103c24c17c288e"
server_key_iv: "b9fc89a66ad750e06b6eb3cdebaa8430"
server_key_version: "encryptionkey@0"
I20260810 18:48:29.382098 22764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance:
uuid: "bd524e1076f2407887e86e92fcb7f717"
format_stamp: "Formatted at 2026-08-10 18:48:29 on dist-test-slave-cbsf"
server_key: "30c413aadee57f0c51103c24c17c288e"
server_key_iv: "b9fc89a66ad750e06b6eb3cdebaa8430"
server_key_version: "encryptionkey@0"
I20260810 18:48:29.386111 22764 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:48:29.388500 22782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:29.389772 22764 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:48:29.389889 22764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-1/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-1/wal
uuid: "bd524e1076f2407887e86e92fcb7f717"
format_stamp: "Formatted at 2026-08-10 18:48:29 on dist-test-slave-cbsf"
server_key: "30c413aadee57f0c51103c24c17c288e"
server_key_iv: "b9fc89a66ad750e06b6eb3cdebaa8430"
server_key_version: "encryptionkey@0"
I20260810 18:48:29.389992 22764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:29.415390 22764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:29.418629 22764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:29.418843 22764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:29.419433 22764 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:29.420257 22764 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:29.420302 22764 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:29.420367 22764 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:29.420401 22764 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:29.429245 22764 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.130:38567
I20260810 18:48:29.429278 22908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.130:38567 every 8 connection(s)
I20260810 18:48:29.430084 22764 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
I20260810 18:48:29.434295 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 22764
I20260810 18:48:29.434434 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance
I20260810 18:48:29.434700 19106 external_mini_cluster.cc:1428] Setting key 1aee3980f4cf55267b3a160eeb5602a4
Aug 10 18:48:29 dist-test-slave-cbsf krb5kdc[22491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387709, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:29.444623 22584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:29.431773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:55621 (local address 127.18.168.190:45849)
0810 18:48:29.431903 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:29.431906 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:29.432736 (+ 830us) server_negotiation.cc:407] Connection header received
0810 18:48:29.434352 (+ 1616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:29.434355 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:29.434415 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:29.434528 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:29.435715 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.436251 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.437478 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.437708 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.437818 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:29.440855 (+ 3037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:29.440858 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:29.440860 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:29.440883 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:29.442340 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:29.443004 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:29.443006 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:29.443007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:29.443081 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:29.443497 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:29.443499 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:29.443500 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:29.443683 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:29.443758 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:29.444391 (+ 633us) server_negotiation.cc:299] Negotiation successful
0810 18:48:29.444489 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 18:48:29.445787 22911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:29.432130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:45849 (local address 127.18.168.130:55621)
0810 18:48:29.432563 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:29.432610 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:29.434108 (+ 1498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:29.434711 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:29.434717 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:29.435099 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:29.435558 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:29.435570 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.436382 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.436386 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:29.437342 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:29.437352 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.437855 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.437860 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:29.438086 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:29.438741 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:29.438767 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:29.440670 (+ 1903us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:29.442460 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:29.442470 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:29.442488 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:29.442894 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:29.443183 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:29.443188 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:29.443192 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:29.443393 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:29.443790 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:29.443799 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:29.444121 (+ 322us) client_negotiation.cc:769] Sending connection context
0810 18:48:29.444343 (+ 222us) client_negotiation.cc:240] Negotiation successful
0810 18:48:29.444726 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":111,"threads_started":1}
WARNING: no policy specified for kudu/127.18.168.131@KRBTEST.COM; defaulting to no policy
I20260810 18:48:29.447024 22909 heartbeater.cc:347] Connected to a master server at 127.18.168.190:45849
I20260810 18:48:29.447234 22909 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:29.447706 22909 heartbeater.cc:511] Master 127.18.168.190:45849 requested a full tablet report, sending...
I20260810 18:48:29.448827 22541 ts_manager.cc:194] Registered new tserver with Master: bd524e1076f2407887e86e92fcb7f717 (127.18.168.130:38567)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.131@KRBTEST.COM" created.
I20260810 18:48:29.449476 22541 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.130@KRBTEST.COM'} at 127.18.168.130:55621
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab.
Entry for principal kudu/127.18.168.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab.
I20260810 18:48:29.465332 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.131:0
--local_ip_for_outbound_sockets=127.18.168.131
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:45849
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:34587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:29.575884 22914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:29.576115 22914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:29.576154 22914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:29.580111 22914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:29.580163 22914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:29.580262 22914 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.131
I20260810 18:48:29.585650 22914 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:34587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:45849
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22914
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.131
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:29.586575 22914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:29.587502 22914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:29.596846 22922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:29.596874 22920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:29.596936 22914 server_base.cc:1034] running on GCE node
W20260810 18:48:29.596886 22919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:29.597532 22914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:29.598105 22914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:29.599290 22914 hybrid_clock.cc:648] HybridClock initialized: now 1786387709599271 us; error 31 us; skew 500 ppm
Aug 10 18:48:29 dist-test-slave-cbsf krb5kdc[22491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387709, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:29.602401 22914 init.cc:377] Logged in from keytab as kudu/127.18.168.131@KRBTEST.COM (short username kudu)
I20260810 18:48:29.603205 22914 webserver.cc:460] Webserver started at http://127.18.168.131:43295/ using document root <none> and password file <none>
I20260810 18:48:29.603744 22914 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:29.603834 22914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:29.604059 22914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:29.606119 22914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-2/data/instance:
uuid: "868923a6345b4610b414c1a940bc890a"
format_stamp: "Formatted at 2026-08-10 18:48:29 on dist-test-slave-cbsf"
server_key: "ad1744400a118ede85670c1f1e54db46"
server_key_iv: "501be469bbe43e46176784eb846fbafa"
server_key_version: "encryptionkey@0"
I20260810 18:48:29.606729 22914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance:
uuid: "868923a6345b4610b414c1a940bc890a"
format_stamp: "Formatted at 2026-08-10 18:48:29 on dist-test-slave-cbsf"
server_key: "ad1744400a118ede85670c1f1e54db46"
server_key_iv: "501be469bbe43e46176784eb846fbafa"
server_key_version: "encryptionkey@0"
I20260810 18:48:29.610515 22914 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:48:29.613058 22930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:29.614172 22914 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:48:29.614301 22914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-2/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-2/wal
uuid: "868923a6345b4610b414c1a940bc890a"
format_stamp: "Formatted at 2026-08-10 18:48:29 on dist-test-slave-cbsf"
server_key: "ad1744400a118ede85670c1f1e54db46"
server_key_iv: "501be469bbe43e46176784eb846fbafa"
server_key_version: "encryptionkey@0"
I20260810 18:48:29.614403 22914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:29.633157 22914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:29.635893 22914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:29.636104 22914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:29.636732 22914 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:29.637615 22914 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:29.637692 22914 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:29.637765 22914 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:29.637820 22914 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:29.646869 22914 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.131:44273
I20260810 18:48:29.646898 23055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.131:44273 every 8 connection(s)
I20260810 18:48:29.647742 22914 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
I20260810 18:48:29.651033 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 22914
I20260810 18:48:29.651170 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance
I20260810 18:48:29.651538 19106 external_mini_cluster.cc:1428] Setting key 873d6e6a203ba4f4af4d2635347ef16c
Aug 10 18:48:29 dist-test-slave-cbsf krb5kdc[22491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387709, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:29.661969 22584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:29.649683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:58767 (local address 127.18.168.190:45849)
0810 18:48:29.649824 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:29.649828 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:29.650714 (+ 886us) server_negotiation.cc:407] Connection header received
0810 18:48:29.652495 (+ 1781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:29.652499 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:29.652558 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:29.652639 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:29.654106 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.654715 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.655909 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.656122 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.656207 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:29.658694 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:29.658698 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:29.658700 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:29.658725 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:29.659949 (+ 1224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:29.660397 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:29.660399 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:29.660401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:29.660473 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:29.660854 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:29.660858 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:29.660860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:29.661066 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:29.661185 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:29.661675 (+ 490us) server_negotiation.cc:299] Negotiation successful
0810 18:48:29.661800 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 18:48:29.662751 23058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:29.650039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:45849 (local address 127.18.168.131:58767)
0810 18:48:29.650496 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:29.650549 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:29.652244 (+ 1695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:29.652872 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:29.652878 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:29.653316 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:29.653950 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:29.653966 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.654827 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.654830 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:29.655804 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:29.655814 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.656251 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.656254 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:29.656397 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:29.657054 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:29.657070 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:29.658539 (+ 1469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:29.660054 (+ 1515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:29.660059 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:29.660070 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:29.660312 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:29.660562 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:29.660564 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:29.660565 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:29.660750 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:29.661180 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:29.661187 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:29.661424 (+ 237us) client_negotiation.cc:769] Sending connection context
0810 18:48:29.661614 (+ 190us) client_negotiation.cc:240] Negotiation successful
0810 18:48:29.661920 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":132,"threads_started":1}
I20260810 18:48:29.663862 23056 heartbeater.cc:347] Connected to a master server at 127.18.168.190:45849
I20260810 18:48:29.664110 23056 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:29.664803 23056 heartbeater.cc:511] Master 127.18.168.190:45849 requested a full tablet report, sending...
I20260810 18:48:29.665993 22541 ts_manager.cc:194] Registered new tserver with Master: 868923a6345b4610b414c1a940bc890a (127.18.168.131:44273)
I20260810 18:48:29.666563 22541 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.131@KRBTEST.COM'} at 127.18.168.131:58767
I20260810 18:48:29.677495 19106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 18:48:29 dist-test-slave-cbsf krb5kdc[22491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387709, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 10 18:48:29 dist-test-slave-cbsf krb5kdc[22491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387709, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:29.704109 22584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:29.696091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43398 (local address 127.18.168.190:45849)
0810 18:48:29.696287 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:29.696291 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:29.696357 (+ 66us) server_negotiation.cc:407] Connection header received
0810 18:48:29.696465 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:29.696467 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:29.696511 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:29.696581 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:29.697323 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.697841 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.698626 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.698831 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.698907 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:29.701368 (+ 2461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:29.701371 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:29.701373 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:29.701395 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:29.702632 (+ 1237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:29.703001 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:29.703004 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:29.703005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:29.703077 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:29.703300 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:29.703303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:29.703304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:29.703510 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:29.703612 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:29.703800 (+ 188us) server_negotiation.cc:299] Negotiation successful
0810 18:48:29.703890 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
W20260810 18:48:29.704505 22541 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:43398
I20260810 18:48:29.705451 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 22612
I20260810 18:48:29.717581 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 22764
I20260810 18:48:29.731196 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 22914
I20260810 18:48:29.738117 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 22504
2026-08-10T18:48:29Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2962 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:48:29 dist-test-slave-cbsf krb5kdc[23070](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:48:29 dist-test-slave-cbsf krb5kdc[23070](info): set up 2 sockets
Aug 10 18:48:29 dist-test-slave-cbsf krb5kdc[23070](info): commencing operation
krb5kdc: starting...
W20260810 18:48:31.005653 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260810 18:48:31.233749 19106 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.228s user 0.001s sys 0.002s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260810 18:48:31.551352 19106 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.317s user 0.001s sys 0.001s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260810 18:48:31.753173 19106 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.202s user 0.000s sys 0.003s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/krb5kdc/test-user.keytab.
W20260810 18:48:31.999975 19106 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.247s user 0.001s sys 0.002s
Aug 10 18:48:32 dist-test-slave-cbsf krb5kdc[23070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:48:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:48:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.168.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
I20260810 18:48:32.052853 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:42083
--webserver_interface=127.18.168.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:42699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:42083
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:32.170145 23085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:32.170392 23085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:32.170432 23085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:32.174408 23085 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:48:32.174463 23085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:32.174482 23085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:32.174533 23085 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:48:32.174566 23085 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:48:32.179802 23085 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:42699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:42083
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:42083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:32.180923 23085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:32.182022 23085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:32.190940 23091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:32.191032 23096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:32.190984 23092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:32.191098 23085 server_base.cc:1034] running on GCE node
I20260810 18:48:32.191892 23085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:32.193099 23085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:32.194310 23085 hybrid_clock.cc:648] HybridClock initialized: now 1786387712194298 us; error 25 us; skew 500 ppm
Aug 10 18:48:32 dist-test-slave-cbsf krb5kdc[23070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387712, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:32.197304 23085 init.cc:377] Logged in from keytab as kudu/127.18.168.190@KRBTEST.COM (short username kudu)
I20260810 18:48:32.198262 23085 webserver.cc:460] Webserver started at http://127.18.168.190:39875/ using document root <none> and password file <none>
I20260810 18:48:32.198868 23085 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:32.198979 23085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:32.199239 23085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:32.201509 23085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "2be281e4925b444c984f04b9028d6f2a"
format_stamp: "Formatted at 2026-08-10 18:48:32 on dist-test-slave-cbsf"
server_key: "c694fcc171d9083b5f1a2718447d0bab"
server_key_iv: "259c4321e6404f1fd4433e12ef45fcbd"
server_key_version: "encryptionkey@0"
I20260810 18:48:32.202096 23085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "2be281e4925b444c984f04b9028d6f2a"
format_stamp: "Formatted at 2026-08-10 18:48:32 on dist-test-slave-cbsf"
server_key: "c694fcc171d9083b5f1a2718447d0bab"
server_key_iv: "259c4321e6404f1fd4433e12ef45fcbd"
server_key_version: "encryptionkey@0"
I20260810 18:48:32.205919 23085 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:48:32.208431 23105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:32.209715 23085 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 18:48:32.209868 23085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "2be281e4925b444c984f04b9028d6f2a"
format_stamp: "Formatted at 2026-08-10 18:48:32 on dist-test-slave-cbsf"
server_key: "c694fcc171d9083b5f1a2718447d0bab"
server_key_iv: "259c4321e6404f1fd4433e12ef45fcbd"
server_key_version: "encryptionkey@0"
I20260810 18:48:32.209981 23085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:32.226377 23085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:32.229256 23085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:32.229476 23085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:32.236925 23085 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:42083
I20260810 18:48:32.236954 23165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:42083 every 8 connection(s)
I20260810 18:48:32.237907 23085 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:48:32.240609 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 23085
I20260810 18:48:32.240772 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:48:32.240813 23166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:32.241109 19106 external_mini_cluster.cc:1428] Setting key ecbed6eb5bf3221175300d326e572181
I20260810 18:48:32.247851 23166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2be281e4925b444c984f04b9028d6f2a: Bootstrap starting.
Aug 10 18:48:32 dist-test-slave-cbsf krb5kdc[23070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:32.250721 23166 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2be281e4925b444c984f04b9028d6f2a: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:32.251731 23166 log.cc:824] T 00000000000000000000000000000000 P 2be281e4925b444c984f04b9028d6f2a: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:32.254247 23166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2be281e4925b444c984f04b9028d6f2a: No bootstrap required, opened a new log
I20260810 18:48:32.255846 23170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:32.242611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53728 (local address 127.18.168.190:42083)
0810 18:48:32.243072 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:32.243088 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:32.243124 (+ 36us) server_negotiation.cc:407] Connection header received
0810 18:48:32.243749 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:32.243766 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:32.244082 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:32.244424 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:32.245321 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:32.246335 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:32.247218 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:32.247706 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:32.247888 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:32.250815 (+ 2927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:32.250824 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:32.250844 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:32.250877 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:32.252611 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:32.253208 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:32.253214 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:32.253219 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:32.253341 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:32.253658 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:32.253660 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:32.253661 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:32.254056 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:32.254203 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:32.254572 (+ 369us) server_negotiation.cc:299] Negotiation successful
0810 18:48:32.254884 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":138,"threads_started":1}
I20260810 18:48:32.258060 23166 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2be281e4925b444c984f04b9028d6f2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2be281e4925b444c984f04b9028d6f2a" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 42083 } }
I20260810 18:48:32.258332 23166 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2be281e4925b444c984f04b9028d6f2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:32.258415 23166 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2be281e4925b444c984f04b9028d6f2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2be281e4925b444c984f04b9028d6f2a, State: Initialized, Role: FOLLOWER
I20260810 18:48:32.258919 23166 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2be281e4925b444c984f04b9028d6f2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2be281e4925b444c984f04b9028d6f2a" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 42083 } }
I20260810 18:48:32.259078 23166 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2be281e4925b444c984f04b9028d6f2a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:48:32.259147 23166 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2be281e4925b444c984f04b9028d6f2a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:48:32.259274 23166 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2be281e4925b444c984f04b9028d6f2a [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:32.260232 23166 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2be281e4925b444c984f04b9028d6f2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2be281e4925b444c984f04b9028d6f2a" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 42083 } }
I20260810 18:48:32.260619 23166 leader_election.cc:302] T 00000000000000000000000000000000 P 2be281e4925b444c984f04b9028d6f2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2be281e4925b444c984f04b9028d6f2a; no voters:
I20260810 18:48:32.260993 23166 leader_election.cc:288] T 00000000000000000000000000000000 P 2be281e4925b444c984f04b9028d6f2a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:48:32.261133 23172 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2be281e4925b444c984f04b9028d6f2a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:32.261363 23172 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2be281e4925b444c984f04b9028d6f2a [term 1 LEADER]: Becoming Leader. State: Replica: 2be281e4925b444c984f04b9028d6f2a, State: Running, Role: LEADER
I20260810 18:48:32.261739 23172 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2be281e4925b444c984f04b9028d6f2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2be281e4925b444c984f04b9028d6f2a" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 42083 } }
I20260810 18:48:32.262285 23166 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2be281e4925b444c984f04b9028d6f2a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:48:32.263373 23174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2be281e4925b444c984f04b9028d6f2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2be281e4925b444c984f04b9028d6f2a. Latest consensus state: current_term: 1 leader_uuid: "2be281e4925b444c984f04b9028d6f2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2be281e4925b444c984f04b9028d6f2a" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 42083 } } }
I20260810 18:48:32.263526 23173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2be281e4925b444c984f04b9028d6f2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2be281e4925b444c984f04b9028d6f2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2be281e4925b444c984f04b9028d6f2a" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 42083 } } }
I20260810 18:48:32.263622 23173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2be281e4925b444c984f04b9028d6f2a [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:32.263622 23174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2be281e4925b444c984f04b9028d6f2a [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:32.264250 23183 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:48:32.267096 23183 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:48:32.272392 23183 catalog_manager.cc:1347] Generated new cluster ID: b81a8a6fdea84018b051d626e43a02f3
I20260810 18:48:32.272454 23183 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:48:32.282814 23183 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:48:32.284108 23183 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:48:32.300335 23183 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2be281e4925b444c984f04b9028d6f2a: Generated new TSK 0
I20260810 18:48:32.301184 23183 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.168.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab.
Entry for principal kudu/127.18.168.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab.
I20260810 18:48:32.336016 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.129:0
--local_ip_for_outbound_sockets=127.18.168.129
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:42083
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:42699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:32.448933 23198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:32.449180 23198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:32.449222 23198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:32.453218 23198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:32.453269 23198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:32.453369 23198 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.129
I20260810 18:48:32.458515 23198 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:42699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:42083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23198
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.129
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:32.459558 23198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:32.460610 23198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:32.470639 23206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:32.470745 23198 server_base.cc:1034] running on GCE node
W20260810 18:48:32.470678 23203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:32.470665 23204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:32.471292 23198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:32.471959 23198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:32.473153 23198 hybrid_clock.cc:648] HybridClock initialized: now 1786387712473140 us; error 25 us; skew 500 ppm
Aug 10 18:48:32 dist-test-slave-cbsf krb5kdc[23070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387712, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:32.475955 23198 init.cc:377] Logged in from keytab as kudu/127.18.168.129@KRBTEST.COM (short username kudu)
I20260810 18:48:32.476905 23198 webserver.cc:460] Webserver started at http://127.18.168.129:40053/ using document root <none> and password file <none>
I20260810 18:48:32.477468 23198 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:32.477536 23198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:32.477770 23198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:32.479768 23198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-0/data/instance:
uuid: "2f3de75193e645cfbe4b4c24cf69e1e1"
format_stamp: "Formatted at 2026-08-10 18:48:32 on dist-test-slave-cbsf"
server_key: "7f239d7d89f15a08463f9ae2dbb3b920"
server_key_iv: "6f8e764dce7195b1907ad2d05da1d7df"
server_key_version: "encryptionkey@0"
I20260810 18:48:32.480327 23198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance:
uuid: "2f3de75193e645cfbe4b4c24cf69e1e1"
format_stamp: "Formatted at 2026-08-10 18:48:32 on dist-test-slave-cbsf"
server_key: "7f239d7d89f15a08463f9ae2dbb3b920"
server_key_iv: "6f8e764dce7195b1907ad2d05da1d7df"
server_key_version: "encryptionkey@0"
I20260810 18:48:32.484302 23198 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260810 18:48:32.486734 23215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:32.487921 23198 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 18:48:32.488065 23198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-0/wal
uuid: "2f3de75193e645cfbe4b4c24cf69e1e1"
format_stamp: "Formatted at 2026-08-10 18:48:32 on dist-test-slave-cbsf"
server_key: "7f239d7d89f15a08463f9ae2dbb3b920"
server_key_iv: "6f8e764dce7195b1907ad2d05da1d7df"
server_key_version: "encryptionkey@0"
I20260810 18:48:32.488168 23198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:32.519956 23198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:32.523247 23198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:32.523458 23198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:32.524077 23198 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:32.524974 23198 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:32.525024 23198 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:32.525087 23198 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:32.525120 23198 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:32.535615 23198 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.129:44943
I20260810 18:48:32.535683 23346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.129:44943 every 8 connection(s)
I20260810 18:48:32.536537 23198 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
I20260810 18:48:32.543069 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 23198
I20260810 18:48:32.543246 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance
I20260810 18:48:32.543640 19106 external_mini_cluster.cc:1428] Setting key 5509b757a3db70226c15b0c8f199930a
Aug 10 18:48:32 dist-test-slave-cbsf krb5kdc[23070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387712, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:32.555220 23170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:32.538603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:43691 (local address 127.18.168.190:42083)
0810 18:48:32.538770 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:32.538776 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:32.539667 (+ 891us) server_negotiation.cc:407] Connection header received
0810 18:48:32.541711 (+ 2044us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:32.541714 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:32.541760 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:32.541835 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:32.543495 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:32.544250 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:32.546356 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:32.546787 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:32.546924 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:32.550276 (+ 3352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:32.550282 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:32.550287 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:32.550322 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:32.552285 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:32.553052 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:32.553055 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:32.553057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:32.553165 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:32.553653 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:32.553658 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:32.553661 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:32.553944 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:32.554062 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:32.554812 (+ 750us) server_negotiation.cc:299] Negotiation successful
0810 18:48:32.554968 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 18:48:32.556293 23350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:32.538948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:42083 (local address 127.18.168.129:43691)
0810 18:48:32.539468 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:32.539527 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:32.541484 (+ 1957us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:32.542025 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:32.542036 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:32.542546 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:32.543266 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:32.543288 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:32.544463 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:32.544470 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:32.546205 (+ 1735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:32.546221 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:32.546982 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:32.546989 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:32.547238 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:32.548018 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:32.548045 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:32.550066 (+ 2021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:32.552444 (+ 2378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:32.552452 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:32.552470 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:32.552913 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:32.553293 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:32.553299 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:32.553301 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:32.553525 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:32.554085 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:32.554094 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:32.554462 (+ 368us) client_negotiation.cc:769] Sending connection context
0810 18:48:32.554716 (+ 254us) client_negotiation.cc:240] Negotiation successful
0810 18:48:32.555088 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":119,"threads_started":1}
WARNING: no policy specified for kudu/127.18.168.130@KRBTEST.COM; defaulting to no policy
I20260810 18:48:32.557832 23347 heartbeater.cc:347] Connected to a master server at 127.18.168.190:42083
I20260810 18:48:32.558073 23347 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:32.558656 23347 heartbeater.cc:511] Master 127.18.168.190:42083 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.130@KRBTEST.COM" created.
I20260810 18:48:32.560580 23126 ts_manager.cc:194] Registered new tserver with Master: 2f3de75193e645cfbe4b4c24cf69e1e1 (127.18.168.129:44943)
I20260810 18:48:32.562136 23126 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.129@KRBTEST.COM'} at 127.18.168.129:43691
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab.
Entry for principal kudu/127.18.168.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab.
I20260810 18:48:32.578208 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.130:0
--local_ip_for_outbound_sockets=127.18.168.130
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:42083
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:42699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:32.691061 23353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:32.691294 23353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:32.691368 23353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:32.695598 23353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:32.695657 23353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:32.695712 23353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.130
I20260810 18:48:32.700855 23353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:42699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:42083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.130
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:32.701853 23353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:32.702833 23353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:32.712460 23360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:32.712532 23359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:32.712584 23353 server_base.cc:1034] running on GCE node
W20260810 18:48:32.712464 23362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:32.713156 23353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:32.713860 23353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:32.715052 23353 hybrid_clock.cc:648] HybridClock initialized: now 1786387712715042 us; error 26 us; skew 500 ppm
Aug 10 18:48:32 dist-test-slave-cbsf krb5kdc[23070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387712, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:32.717835 23353 init.cc:377] Logged in from keytab as kudu/127.18.168.130@KRBTEST.COM (short username kudu)
I20260810 18:48:32.718662 23353 webserver.cc:460] Webserver started at http://127.18.168.130:35521/ using document root <none> and password file <none>
I20260810 18:48:32.719213 23353 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:32.719304 23353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:32.719537 23353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:32.721565 23353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-1/data/instance:
uuid: "5971f3fafaba4c2b8a8133e492cf06a2"
format_stamp: "Formatted at 2026-08-10 18:48:32 on dist-test-slave-cbsf"
server_key: "171055e2677c29b11f059ef1d4d9fb04"
server_key_iv: "63cf1a683fc126b20647333dc260c7df"
server_key_version: "encryptionkey@0"
I20260810 18:48:32.722138 23353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance:
uuid: "5971f3fafaba4c2b8a8133e492cf06a2"
format_stamp: "Formatted at 2026-08-10 18:48:32 on dist-test-slave-cbsf"
server_key: "171055e2677c29b11f059ef1d4d9fb04"
server_key_iv: "63cf1a683fc126b20647333dc260c7df"
server_key_version: "encryptionkey@0"
I20260810 18:48:32.725771 23353 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:48:32.728091 23370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:32.729277 23353 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:48:32.729421 23353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-1/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-1/wal
uuid: "5971f3fafaba4c2b8a8133e492cf06a2"
format_stamp: "Formatted at 2026-08-10 18:48:32 on dist-test-slave-cbsf"
server_key: "171055e2677c29b11f059ef1d4d9fb04"
server_key_iv: "63cf1a683fc126b20647333dc260c7df"
server_key_version: "encryptionkey@0"
I20260810 18:48:32.729527 23353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:32.738524 23353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:32.741550 23353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:32.741747 23353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:32.742305 23353 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:32.743095 23353 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:32.743167 23353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:32.743232 23353 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:32.743299 23353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:32.751914 23353 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.130:45081
I20260810 18:48:32.751942 23499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.130:45081 every 8 connection(s)
I20260810 18:48:32.752826 23353 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
I20260810 18:48:32.755345 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 23353
I20260810 18:48:32.755470 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance
I20260810 18:48:32.755800 19106 external_mini_cluster.cc:1428] Setting key 3d3a7fc84d56039b352fb4dbfef3d12e
Aug 10 18:48:32 dist-test-slave-cbsf krb5kdc[23070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387712, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.168.131@KRBTEST.COM; defaulting to no policy
I20260810 18:48:32.766811 23170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:32.754556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:57939 (local address 127.18.168.190:42083)
0810 18:48:32.754684 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:32.754688 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:32.755679 (+ 991us) server_negotiation.cc:407] Connection header received
0810 18:48:32.757270 (+ 1591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:32.757274 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:32.757329 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:32.757403 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:32.758506 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:32.759028 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:32.760174 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:32.760426 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:32.760549 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:32.763170 (+ 2621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:32.763174 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:32.763177 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:32.763208 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:32.764802 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:32.765292 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:32.765297 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:32.765299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:32.765382 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:32.765733 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:32.765736 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:32.765738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:32.765922 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:32.766036 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:32.766503 (+ 467us) server_negotiation.cc:299] Negotiation successful
0810 18:48:32.766640 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 18:48:32.767853 23502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:32.754952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:42083 (local address 127.18.168.130:57939)
0810 18:48:32.755516 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:32.755568 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:32.757006 (+ 1438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:32.757566 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:32.757573 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:32.757896 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:32.758365 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:32.758377 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:32.759179 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:32.759188 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:32.760038 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:32.760048 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:32.760578 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:32.760583 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:32.760828 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:32.761415 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:32.761431 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:32.762898 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:32.764912 (+ 2014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:32.764917 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:32.764931 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:32.765186 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:32.765482 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:32.765485 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:32.765487 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:32.765634 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:32.766028 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:32.766034 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:32.766273 (+ 239us) client_negotiation.cc:769] Sending connection context
0810 18:48:32.766460 (+ 187us) client_negotiation.cc:240] Negotiation successful
0810 18:48:32.766806 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":156,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.131@KRBTEST.COM" created.
I20260810 18:48:32.769192 23500 heartbeater.cc:347] Connected to a master server at 127.18.168.190:42083
I20260810 18:48:32.769435 23500 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:32.769930 23500 heartbeater.cc:511] Master 127.18.168.190:42083 requested a full tablet report, sending...
I20260810 18:48:32.771059 23126 ts_manager.cc:194] Registered new tserver with Master: 5971f3fafaba4c2b8a8133e492cf06a2 (127.18.168.130:45081)
I20260810 18:48:32.771939 23126 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.130@KRBTEST.COM'} at 127.18.168.130:57939
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab.
Entry for principal kudu/127.18.168.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab.
I20260810 18:48:32.784238 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.131:0
--local_ip_for_outbound_sockets=127.18.168.131
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:42083
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:42699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:32.895462 23505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:32.895720 23505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:32.895762 23505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:32.899789 23505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:32.899848 23505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:32.899906 23505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.131
I20260810 18:48:32.905083 23505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:42699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:42083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.131
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:32.906069 23505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:32.907043 23505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:32.917006 23512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:32.917021 23515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:32.917096 23505 server_base.cc:1034] running on GCE node
W20260810 18:48:32.917014 23511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:32.917712 23505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:32.918310 23505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:32.919509 23505 hybrid_clock.cc:648] HybridClock initialized: now 1786387712919482 us; error 37 us; skew 500 ppm
Aug 10 18:48:32 dist-test-slave-cbsf krb5kdc[23070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387712, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:32.922308 23505 init.cc:377] Logged in from keytab as kudu/127.18.168.131@KRBTEST.COM (short username kudu)
I20260810 18:48:32.923108 23505 webserver.cc:460] Webserver started at http://127.18.168.131:37709/ using document root <none> and password file <none>
I20260810 18:48:32.923648 23505 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:32.923707 23505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:32.923864 23505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:32.925894 23505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-2/data/instance:
uuid: "948898bd29744a859a97443c55a8d8b1"
format_stamp: "Formatted at 2026-08-10 18:48:32 on dist-test-slave-cbsf"
server_key: "accd6af33f60d3373b06467b406e5a70"
server_key_iv: "4d2a3fbbd795d10210fbae3f8529b545"
server_key_version: "encryptionkey@0"
I20260810 18:48:32.926381 23505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance:
uuid: "948898bd29744a859a97443c55a8d8b1"
format_stamp: "Formatted at 2026-08-10 18:48:32 on dist-test-slave-cbsf"
server_key: "accd6af33f60d3373b06467b406e5a70"
server_key_iv: "4d2a3fbbd795d10210fbae3f8529b545"
server_key_version: "encryptionkey@0"
I20260810 18:48:32.929880 23505 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:48:32.932098 23522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:32.933382 23505 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:48:32.933517 23505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-2/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-2/wal
uuid: "948898bd29744a859a97443c55a8d8b1"
format_stamp: "Formatted at 2026-08-10 18:48:32 on dist-test-slave-cbsf"
server_key: "accd6af33f60d3373b06467b406e5a70"
server_key_iv: "4d2a3fbbd795d10210fbae3f8529b545"
server_key_version: "encryptionkey@0"
I20260810 18:48:32.933620 23505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:32.957114 23505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:32.960428 23505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:32.960594 23505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:32.961275 23505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:32.962163 23505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:32.962211 23505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:32.962281 23505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:32.962320 23505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:32.971479 23505 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.131:38291
I20260810 18:48:32.971506 23650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.131:38291 every 8 connection(s)
I20260810 18:48:32.972364 23505 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
I20260810 18:48:32.981096 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 23505
I20260810 18:48:32.981266 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance
I20260810 18:48:32.981556 19106 external_mini_cluster.cc:1428] Setting key 86e740d9154af91d112c6c516a44705a
Aug 10 18:48:32 dist-test-slave-cbsf krb5kdc[23070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387712, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:32.986632 23170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:32.974203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:59397 (local address 127.18.168.190:42083)
0810 18:48:32.974359 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:32.974363 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:32.975206 (+ 843us) server_negotiation.cc:407] Connection header received
0810 18:48:32.976611 (+ 1405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:32.976615 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:32.976686 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:32.976808 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:32.978042 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:32.978592 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:32.979767 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:32.979997 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:32.980117 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:32.982992 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:32.982996 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:32.982999 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:32.983029 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:32.984744 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:32.985229 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:32.985232 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:32.985233 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:32.985322 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:32.985635 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:32.985638 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:32.985640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:32.985821 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:32.985899 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:32.986396 (+ 497us) server_negotiation.cc:299] Negotiation successful
0810 18:48:32.986512 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 18:48:32.987444 23653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:32.974593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:42083 (local address 127.18.168.131:59397)
0810 18:48:32.975046 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:32.975089 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:32.976386 (+ 1297us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:32.976989 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:32.976995 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:32.977368 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:32.977880 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:32.977893 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:32.978733 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:32.978736 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:32.979630 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:32.979640 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:32.980167 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:32.980172 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:32.980345 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:32.981003 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:32.981020 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:32.982829 (+ 1809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:32.984850 (+ 2021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:32.984855 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:32.984868 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:32.985132 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:32.985409 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:32.985411 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:32.985413 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:32.985553 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:32.985902 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:32.985907 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:32.986150 (+ 243us) client_negotiation.cc:769] Sending connection context
0810 18:48:32.986314 (+ 164us) client_negotiation.cc:240] Negotiation successful
0810 18:48:32.986592 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":111,"threads_started":1}
I20260810 18:48:32.988657 23651 heartbeater.cc:347] Connected to a master server at 127.18.168.190:42083
I20260810 18:48:32.988977 23651 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:32.989496 23651 heartbeater.cc:511] Master 127.18.168.190:42083 requested a full tablet report, sending...
I20260810 18:48:32.990521 23126 ts_manager.cc:194] Registered new tserver with Master: 948898bd29744a859a97443c55a8d8b1 (127.18.168.131:38291)
I20260810 18:48:32.991093 23126 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.131@KRBTEST.COM'} at 127.18.168.131:59397
I20260810 18:48:32.996161 19106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 18:48:33 dist-test-slave-cbsf krb5kdc[23070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 18:48:33 dist-test-slave-cbsf krb5kdc[23070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.168.129@KRBTEST.COM
I20260810 18:48:33.024322 23659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:33.014997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38882 (local address 127.18.168.129:44943)
0810 18:48:33.015316 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:33.015332 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:33.015356 (+ 24us) server_negotiation.cc:407] Connection header received
0810 18:48:33.015409 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:33.015414 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:33.015599 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:33.015726 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:33.016351 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:33.017249 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:33.018053 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:33.018421 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:33.018502 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:33.021229 (+ 2727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:33.021238 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:33.021252 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:33.021272 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:33.022763 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:33.023183 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:33.023187 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:33.023189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:33.023278 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:33.023525 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:33.023527 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:33.023528 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:33.023756 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:33.023914 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:33.024049 (+ 135us) server_negotiation.cc:299] Negotiation successful
0810 18:48:33.024178 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":116,"threads_started":1}
Aug 10 18:48:33 dist-test-slave-cbsf krb5kdc[23070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:33.035856 23170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:33.027206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53738 (local address 127.18.168.190:42083)
0810 18:48:33.027352 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:33.027357 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:33.027414 (+ 57us) server_negotiation.cc:407] Connection header received
0810 18:48:33.027577 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:33.027579 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:33.027621 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:33.027704 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:33.028271 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:33.028828 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:33.029567 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:33.029802 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:33.029909 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:33.032082 (+ 2173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:33.032086 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:33.032087 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:33.032110 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:33.033653 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:33.034037 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:33.034040 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:33.034041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:33.034137 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:33.034559 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:33.034562 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:33.034563 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:33.034909 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:33.035078 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:33.035458 (+ 380us) server_negotiation.cc:299] Negotiation successful
0810 18:48:33.035626 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260810 18:48:33.036337 23126 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53738
I20260810 18:48:33.037051 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 23198
I20260810 18:48:33.051149 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 23353
I20260810 18:48:33.059578 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 23505
I20260810 18:48:33.067068 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 23085
2026-08-10T18:48:33Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3329 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:48:33 dist-test-slave-cbsf krb5kdc[23664](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:48:33 dist-test-slave-cbsf krb5kdc[23664](info): set up 2 sockets
Aug 10 18:48:33 dist-test-slave-cbsf krb5kdc[23664](info): commencing operation
krb5kdc: starting...
W20260810 18:48:34.189868 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:48:34 dist-test-slave-cbsf krb5kdc[23664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:48:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:48:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.168.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
I20260810 18:48:34.319561 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:41655
--webserver_interface=127.18.168.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:43519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:41655
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:34.428822 23680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:34.429065 23680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:34.429106 23680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:34.432965 23680 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:48:34.433010 23680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:34.433033 23680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:34.433049 23680 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:48:34.433063 23680 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:48:34.437989 23680 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:43519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:41655
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:41655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23680
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:34.438887 23680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:34.439774 23680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:34.448055 23688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:34.448092 23691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:34.448089 23687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:34.448231 23680 server_base.cc:1034] running on GCE node
I20260810 18:48:34.449002 23680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:34.450011 23680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:34.451182 23680 hybrid_clock.cc:648] HybridClock initialized: now 1786387714451164 us; error 30 us; skew 500 ppm
Aug 10 18:48:34 dist-test-slave-cbsf krb5kdc[23664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387714, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:34.457275 23680 init.cc:377] Logged in from keytab as kudu/127.18.168.190@KRBTEST.COM (short username kudu)
I20260810 18:48:34.458014 23680 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:34.458112 23680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:34.458369 23680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:34.460376 23680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "c2305433a4634510925ef775003a48bd"
format_stamp: "Formatted at 2026-08-10 18:48:34 on dist-test-slave-cbsf"
server_key: "ee23383410428aba9720539b40dc177b"
server_key_iv: "e0a2d3d0a0b1a71bf892637f82316d5e"
server_key_version: "encryptionkey@0"
I20260810 18:48:34.460959 23680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "c2305433a4634510925ef775003a48bd"
format_stamp: "Formatted at 2026-08-10 18:48:34 on dist-test-slave-cbsf"
server_key: "ee23383410428aba9720539b40dc177b"
server_key_iv: "e0a2d3d0a0b1a71bf892637f82316d5e"
server_key_version: "encryptionkey@0"
I20260810 18:48:34.464653 23680 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 18:48:34.467103 23698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:34.468292 23680 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:48:34.468441 23680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "c2305433a4634510925ef775003a48bd"
format_stamp: "Formatted at 2026-08-10 18:48:34 on dist-test-slave-cbsf"
server_key: "ee23383410428aba9720539b40dc177b"
server_key_iv: "e0a2d3d0a0b1a71bf892637f82316d5e"
server_key_version: "encryptionkey@0"
I20260810 18:48:34.468550 23680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:34.485173 23680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:34.487951 23680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:34.488163 23680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:34.495205 23680 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:41655
I20260810 18:48:34.495493 23766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:41655 every 8 connection(s)
I20260810 18:48:34.495870 23680 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:48:34.498597 23767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:34.503891 23767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2305433a4634510925ef775003a48bd: Bootstrap starting.
I20260810 18:48:34.505514 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 23680
I20260810 18:48:34.505659 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:48:34.505918 19106 external_mini_cluster.cc:1428] Setting key c409121e3a68a090bd0a79b16af63d51
I20260810 18:48:34.506415 23767 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2305433a4634510925ef775003a48bd: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:34.507264 23767 log.cc:824] T 00000000000000000000000000000000 P c2305433a4634510925ef775003a48bd: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:34.509236 23767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2305433a4634510925ef775003a48bd: No bootstrap required, opened a new log
I20260810 18:48:34.513336 23767 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2305433a4634510925ef775003a48bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2305433a4634510925ef775003a48bd" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41655 } }
I20260810 18:48:34.513625 23767 raft_consensus.cc:374] T 00000000000000000000000000000000 P c2305433a4634510925ef775003a48bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:34.513692 23767 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2305433a4634510925ef775003a48bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2305433a4634510925ef775003a48bd, State: Initialized, Role: FOLLOWER
I20260810 18:48:34.514199 23767 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2305433a4634510925ef775003a48bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2305433a4634510925ef775003a48bd" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41655 } }
I20260810 18:48:34.514348 23767 raft_consensus.cc:388] T 00000000000000000000000000000000 P c2305433a4634510925ef775003a48bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:48:34.514446 23767 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2305433a4634510925ef775003a48bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:48:34.514565 23767 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2305433a4634510925ef775003a48bd [term 0 FOLLOWER]: Advancing to term 1
Aug 10 18:48:34 dist-test-slave-cbsf krb5kdc[23664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:34.515544 23767 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2305433a4634510925ef775003a48bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2305433a4634510925ef775003a48bd" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41655 } }
I20260810 18:48:34.515865 23767 leader_election.cc:302] T 00000000000000000000000000000000 P c2305433a4634510925ef775003a48bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c2305433a4634510925ef775003a48bd; no voters:
I20260810 18:48:34.516188 23767 leader_election.cc:288] T 00000000000000000000000000000000 P c2305433a4634510925ef775003a48bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:48:34.516259 23773 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c2305433a4634510925ef775003a48bd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:34.516503 23773 raft_consensus.cc:686] T 00000000000000000000000000000000 P c2305433a4634510925ef775003a48bd [term 1 LEADER]: Becoming Leader. State: Replica: c2305433a4634510925ef775003a48bd, State: Running, Role: LEADER
I20260810 18:48:34.516858 23773 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2305433a4634510925ef775003a48bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2305433a4634510925ef775003a48bd" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41655 } }
I20260810 18:48:34.517549 23767 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2305433a4634510925ef775003a48bd [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:48:34.518251 23775 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2305433a4634510925ef775003a48bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c2305433a4634510925ef775003a48bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2305433a4634510925ef775003a48bd" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41655 } } }
I20260810 18:48:34.518384 23775 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2305433a4634510925ef775003a48bd [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:34.518689 23774 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2305433a4634510925ef775003a48bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2305433a4634510925ef775003a48bd. Latest consensus state: current_term: 1 leader_uuid: "c2305433a4634510925ef775003a48bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2305433a4634510925ef775003a48bd" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41655 } } }
I20260810 18:48:34.518846 23774 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2305433a4634510925ef775003a48bd [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:34.521308 23771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:34.507084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52854 (local address 127.18.168.190:41655)
0810 18:48:34.507568 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:34.507589 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:34.507646 (+ 57us) server_negotiation.cc:407] Connection header received
0810 18:48:34.508424 (+ 778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:34.508442 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:34.508822 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:34.509083 (+ 261us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:34.509996 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.511022 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.512263 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.512794 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.512909 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:34.515672 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:34.515679 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:34.515696 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:34.515726 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:34.517641 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:34.518149 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:34.518155 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:34.518161 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:34.518314 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:34.518709 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:34.518712 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:34.518714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:34.519220 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:34.519434 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:34.520241 (+ 807us) server_negotiation.cc:299] Negotiation successful
0810 18:48:34.520543 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":178,"threads_started":1}
W20260810 18:48:34.523859 23792 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c2305433a4634510925ef775003a48bd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:48:34.523940 23792 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:48:34.524078 23784 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:48:34.525257 23784 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:48:34.530308 23784 catalog_manager.cc:1347] Generated new cluster ID: af7654f694904f14bddfc112be8de250
I20260810 18:48:34.530365 23784 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:48:34.555121 23784 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:48:34.556425 23784 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:48:34.559855 23784 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c2305433a4634510925ef775003a48bd: Generated new TSK 0
I20260810 18:48:34.560546 23784 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.168.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab.
Entry for principal kudu/127.18.168.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab.
I20260810 18:48:34.597415 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.129:0
--local_ip_for_outbound_sockets=127.18.168.129
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:41655
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:43519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:34.714344 23803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:34.714569 23803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:34.714609 23803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:34.718814 23803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:34.718864 23803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:34.718961 23803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.129
I20260810 18:48:34.724080 23803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:43519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:41655
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.129
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:34.725160 23803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:34.726169 23803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:34.735621 23809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:34.735623 23811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:34.735606 23808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:34.735855 23803 server_base.cc:1034] running on GCE node
I20260810 18:48:34.736353 23803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:34.737022 23803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:34.738209 23803 hybrid_clock.cc:648] HybridClock initialized: now 1786387714738196 us; error 25 us; skew 500 ppm
Aug 10 18:48:34 dist-test-slave-cbsf krb5kdc[23664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387714, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:34.740963 23803 init.cc:377] Logged in from keytab as kudu/127.18.168.129@KRBTEST.COM (short username kudu)
I20260810 18:48:34.741627 23803 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:34.741688 23803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:34.741845 23803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:34.743763 23803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-0/data/instance:
uuid: "7e5f2a8e152c4e45bd2f2a109bc6a8e9"
format_stamp: "Formatted at 2026-08-10 18:48:34 on dist-test-slave-cbsf"
server_key: "be81b1e25118401f6618d1b6cfa074fc"
server_key_iv: "d52705c66f633189f193a230b12e255b"
server_key_version: "encryptionkey@0"
I20260810 18:48:34.744271 23803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance:
uuid: "7e5f2a8e152c4e45bd2f2a109bc6a8e9"
format_stamp: "Formatted at 2026-08-10 18:48:34 on dist-test-slave-cbsf"
server_key: "be81b1e25118401f6618d1b6cfa074fc"
server_key_iv: "d52705c66f633189f193a230b12e255b"
server_key_version: "encryptionkey@0"
I20260810 18:48:34.747725 23803 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.002s
I20260810 18:48:34.749969 23820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:34.751168 23803 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:48:34.751291 23803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-0/wal
uuid: "7e5f2a8e152c4e45bd2f2a109bc6a8e9"
format_stamp: "Formatted at 2026-08-10 18:48:34 on dist-test-slave-cbsf"
server_key: "be81b1e25118401f6618d1b6cfa074fc"
server_key_iv: "d52705c66f633189f193a230b12e255b"
server_key_version: "encryptionkey@0"
I20260810 18:48:34.751389 23803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:34.762444 23803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:34.765337 23803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:34.765553 23803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:34.766110 23803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:34.766924 23803 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:34.766971 23803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:34.767046 23803 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:34.767076 23803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:34.775978 23803 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.129:44433
I20260810 18:48:34.776051 23949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.129:44433 every 8 connection(s)
I20260810 18:48:34.776574 23803 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
I20260810 18:48:34.784267 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 23803
I20260810 18:48:34.784415 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance
I20260810 18:48:34.784742 19106 external_mini_cluster.cc:1428] Setting key 94ab9bc87b326a354c32fb9ce58a5ed6
Aug 10 18:48:34 dist-test-slave-cbsf krb5kdc[23664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387714, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:34.791020 23771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:34.778294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:46517 (local address 127.18.168.190:41655)
0810 18:48:34.778426 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:34.778431 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:34.779335 (+ 904us) server_negotiation.cc:407] Connection header received
0810 18:48:34.780884 (+ 1549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:34.780887 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:34.780936 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:34.781009 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:34.782113 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.782624 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.783711 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.783925 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.784029 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:34.786946 (+ 2917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:34.786953 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:34.786958 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:34.786991 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:34.788727 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:34.789367 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:34.789370 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:34.789371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:34.789450 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:34.789896 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:34.789898 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:34.789900 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:34.790072 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:34.790149 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:34.790794 (+ 645us) server_negotiation.cc:299] Negotiation successful
0810 18:48:34.790895 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 18:48:34.792217 23952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:34.778672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41655 (local address 127.18.168.129:46517)
0810 18:48:34.779166 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:34.779219 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:34.780564 (+ 1345us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:34.781184 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:34.781191 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:34.781514 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:34.781971 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:34.781983 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.782742 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.782744 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:34.783605 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:34.783614 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.784073 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.784075 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:34.784213 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:34.784812 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:34.784846 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:34.786767 (+ 1921us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:34.788859 (+ 2092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:34.788867 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:34.788883 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:34.789250 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:34.789600 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:34.789605 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:34.789607 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:34.789797 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:34.790180 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:34.790190 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:34.790537 (+ 347us) client_negotiation.cc:769] Sending connection context
0810 18:48:34.790748 (+ 211us) client_negotiation.cc:240] Negotiation successful
0810 18:48:34.791090 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":109,"threads_started":1}
I20260810 18:48:34.793579 23950 heartbeater.cc:347] Connected to a master server at 127.18.168.190:41655
I20260810 18:48:34.793830 23950 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:34.794279 23950 heartbeater.cc:511] Master 127.18.168.190:41655 requested a full tablet report, sending...
I20260810 18:48:34.795802 23718 ts_manager.cc:194] Registered new tserver with Master: 7e5f2a8e152c4e45bd2f2a109bc6a8e9 (127.18.168.129:44433)
WARNING: no policy specified for kudu/127.18.168.130@KRBTEST.COM; defaulting to no policy
I20260810 18:48:34.797048 23718 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.129@KRBTEST.COM'} at 127.18.168.129:46517
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab.
Entry for principal kudu/127.18.168.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab.
I20260810 18:48:34.815567 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.130:0
--local_ip_for_outbound_sockets=127.18.168.130
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:41655
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:43519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:34.925417 23956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:34.925727 23956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:34.925804 23956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:34.929625 23956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:34.929702 23956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:34.929790 23956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.130
I20260810 18:48:34.934896 23956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:43519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:41655
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.130
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:34.935863 23956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:34.936841 23956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:34.945883 23961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:34.945899 23962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:34.945986 23964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:34.946246 23956 server_base.cc:1034] running on GCE node
I20260810 18:48:34.946936 23956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:34.947748 23956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:34.948930 23956 hybrid_clock.cc:648] HybridClock initialized: now 1786387714948897 us; error 47 us; skew 500 ppm
Aug 10 18:48:34 dist-test-slave-cbsf krb5kdc[23664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387714, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:34.952857 23956 init.cc:377] Logged in from keytab as kudu/127.18.168.130@KRBTEST.COM (short username kudu)
I20260810 18:48:34.953802 23956 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:34.953869 23956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:34.954133 23956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:34.956614 23956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-1/data/instance:
uuid: "da7683da45a942cc9e4753af04f2b117"
format_stamp: "Formatted at 2026-08-10 18:48:34 on dist-test-slave-cbsf"
server_key: "494576a9d5ea71d051c2be84bc8d11b5"
server_key_iv: "8501cb602e4536af61d1952ba3e02e54"
server_key_version: "encryptionkey@0"
I20260810 18:48:34.957214 23956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance:
uuid: "da7683da45a942cc9e4753af04f2b117"
format_stamp: "Formatted at 2026-08-10 18:48:34 on dist-test-slave-cbsf"
server_key: "494576a9d5ea71d051c2be84bc8d11b5"
server_key_iv: "8501cb602e4536af61d1952ba3e02e54"
server_key_version: "encryptionkey@0"
I20260810 18:48:34.960820 23956 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:48:34.963104 23971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:34.964290 23956 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:48:34.964425 23956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-1/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-1/wal
uuid: "da7683da45a942cc9e4753af04f2b117"
format_stamp: "Formatted at 2026-08-10 18:48:34 on dist-test-slave-cbsf"
server_key: "494576a9d5ea71d051c2be84bc8d11b5"
server_key_iv: "8501cb602e4536af61d1952ba3e02e54"
server_key_version: "encryptionkey@0"
I20260810 18:48:34.964552 23956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:34.981413 23956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:34.984248 23956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:34.984450 23956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:34.985116 23956 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:34.985970 23956 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:34.986045 23956 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:34.986119 23956 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:34.986172 23956 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:34.996909 23956 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.130:36635
I20260810 18:48:34.996917 24098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.130:36635 every 8 connection(s)
I20260810 18:48:34.997725 23956 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
I20260810 18:48:35.002153 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 23956
I20260810 18:48:35.002287 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance
I20260810 18:48:35.002585 19106 external_mini_cluster.cc:1428] Setting key 636f5c83ffc05bfa7be894ae96a73b9f
Aug 10 18:48:35 dist-test-slave-cbsf krb5kdc[23664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387714, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:35.013401 23771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:34.999735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:50141 (local address 127.18.168.190:41655)
0810 18:48:34.999870 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:34.999877 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:35.001141 (+ 1264us) server_negotiation.cc:407] Connection header received
0810 18:48:35.002759 (+ 1618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:35.002762 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:35.002825 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:35.002910 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:35.004297 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:35.004905 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:35.006091 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:35.006316 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:35.006429 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:35.009274 (+ 2845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:35.009280 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:35.009285 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:35.009318 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:35.011135 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:35.011641 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:35.011644 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:35.011647 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:35.011742 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:35.012072 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:35.012079 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:35.012081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:35.012348 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:35.012465 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:35.013043 (+ 578us) server_negotiation.cc:299] Negotiation successful
0810 18:48:35.013179 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
WARNING: no policy specified for kudu/127.18.168.131@KRBTEST.COM; defaulting to no policy
I20260810 18:48:35.014343 24103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:35.000268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41655 (local address 127.18.168.130:50141)
0810 18:48:35.000955 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:35.001014 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:35.002528 (+ 1514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:35.003091 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:35.003098 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:35.003570 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:35.004141 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:35.004162 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:35.005039 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:35.005043 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:35.005959 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:35.005968 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:35.006474 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:35.006478 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:35.006649 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:35.007301 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:35.007317 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:35.009077 (+ 1760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:35.011259 (+ 2182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:35.011264 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:35.011277 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:35.011534 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:35.011839 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:35.011842 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:35.011843 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:35.011973 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:35.012454 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:35.012459 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:35.012758 (+ 299us) client_negotiation.cc:769] Sending connection context
0810 18:48:35.013004 (+ 246us) client_negotiation.cc:240] Negotiation successful
0810 18:48:35.013364 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":149,"threads_started":1}
I20260810 18:48:35.015620 24101 heartbeater.cc:347] Connected to a master server at 127.18.168.190:41655
I20260810 18:48:35.015874 24101 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:35.016456 24101 heartbeater.cc:511] Master 127.18.168.190:41655 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.131@KRBTEST.COM" created.
I20260810 18:48:35.017724 23718 ts_manager.cc:194] Registered new tserver with Master: da7683da45a942cc9e4753af04f2b117 (127.18.168.130:36635)
I20260810 18:48:35.018379 23718 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.130@KRBTEST.COM'} at 127.18.168.130:50141
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab.
Entry for principal kudu/127.18.168.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab.
I20260810 18:48:35.035550 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.131:0
--local_ip_for_outbound_sockets=127.18.168.131
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:41655
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:43519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:35.153873 24106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:35.154116 24106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:35.154163 24106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:35.158294 24106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:35.158357 24106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:35.158417 24106 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.131
I20260810 18:48:35.163638 24106 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:43519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:41655
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24106
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.131
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:35.164606 24106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:35.165614 24106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:35.175464 24112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:35.175459 24111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:35.175459 24114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:35.175526 24106 server_base.cc:1034] running on GCE node
I20260810 18:48:35.176136 24106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:35.176779 24106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:35.177935 24106 hybrid_clock.cc:648] HybridClock initialized: now 1786387715177923 us; error 24 us; skew 500 ppm
Aug 10 18:48:35 dist-test-slave-cbsf krb5kdc[23664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387715, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:35.180752 24106 init.cc:377] Logged in from keytab as kudu/127.18.168.131@KRBTEST.COM (short username kudu)
I20260810 18:48:35.181449 24106 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:35.181545 24106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:35.181792 24106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:35.183805 24106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-2/data/instance:
uuid: "872d981c457f480faf66381c11d3097a"
format_stamp: "Formatted at 2026-08-10 18:48:35 on dist-test-slave-cbsf"
server_key: "c6501f41974a86c4a9de4e74ec28bd6b"
server_key_iv: "1bc58671da83f16b4d64b4f56140862a"
server_key_version: "encryptionkey@0"
I20260810 18:48:35.184374 24106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance:
uuid: "872d981c457f480faf66381c11d3097a"
format_stamp: "Formatted at 2026-08-10 18:48:35 on dist-test-slave-cbsf"
server_key: "c6501f41974a86c4a9de4e74ec28bd6b"
server_key_iv: "1bc58671da83f16b4d64b4f56140862a"
server_key_version: "encryptionkey@0"
I20260810 18:48:35.188079 24106 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.001s
I20260810 18:48:35.190660 24122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:35.191953 24106 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 18:48:35.192087 24106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-2/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-2/wal
uuid: "872d981c457f480faf66381c11d3097a"
format_stamp: "Formatted at 2026-08-10 18:48:35 on dist-test-slave-cbsf"
server_key: "c6501f41974a86c4a9de4e74ec28bd6b"
server_key_iv: "1bc58671da83f16b4d64b4f56140862a"
server_key_version: "encryptionkey@0"
I20260810 18:48:35.192203 24106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:35.209977 24106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:35.212992 24106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:35.213203 24106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:35.213899 24106 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:35.214766 24106 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:35.214839 24106 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:35.214912 24106 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:35.214956 24106 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:35.224090 24106 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.131:39285
I20260810 18:48:35.224108 24262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.131:39285 every 8 connection(s)
I20260810 18:48:35.224745 24106 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
I20260810 18:48:35.232519 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 24106
I20260810 18:48:35.232672 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance
I20260810 18:48:35.232986 19106 external_mini_cluster.cc:1428] Setting key ec7a356bbd60acee83f4645ec6029741
Aug 10 18:48:35 dist-test-slave-cbsf krb5kdc[23664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387715, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:35.238161 23771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:35.226489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:56343 (local address 127.18.168.190:41655)
0810 18:48:35.226638 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:35.226642 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:35.227477 (+ 835us) server_negotiation.cc:407] Connection header received
0810 18:48:35.228803 (+ 1326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:35.228806 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:35.228865 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:35.228954 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:35.230188 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:35.230735 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:35.231899 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:35.232138 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:35.232255 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:35.235108 (+ 2853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:35.235112 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:35.235115 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:35.235143 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:35.236338 (+ 1195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:35.236843 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:35.236845 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:35.236846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:35.236924 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:35.237255 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:35.237257 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:35.237258 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:35.237423 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:35.237497 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:35.237937 (+ 440us) server_negotiation.cc:299] Negotiation successful
0810 18:48:35.238046 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 18:48:35.238953 24265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:35.226879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41655 (local address 127.18.168.131:56343)
0810 18:48:35.227331 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:35.227373 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:35.228558 (+ 1185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:35.229184 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:35.229191 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:35.229556 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:35.230031 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:35.230045 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:35.230866 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:35.230869 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:35.231765 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:35.231775 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:35.232300 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:35.232304 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:35.232488 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:35.233163 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:35.233187 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:35.234902 (+ 1715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:35.236440 (+ 1538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:35.236444 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:35.236455 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:35.236757 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:35.237004 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:35.237006 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:35.237007 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:35.237135 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:35.237507 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:35.237512 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:35.237729 (+ 217us) client_negotiation.cc:769] Sending connection context
0810 18:48:35.237875 (+ 146us) client_negotiation.cc:240] Negotiation successful
0810 18:48:35.238106 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":114,"threads_started":1}
I20260810 18:48:35.240002 24263 heartbeater.cc:347] Connected to a master server at 127.18.168.190:41655
I20260810 18:48:35.240229 24263 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:35.240656 24263 heartbeater.cc:511] Master 127.18.168.190:41655 requested a full tablet report, sending...
I20260810 18:48:35.241715 23718 ts_manager.cc:194] Registered new tserver with Master: 872d981c457f480faf66381c11d3097a (127.18.168.131:39285)
I20260810 18:48:35.242277 23718 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.131@KRBTEST.COM'} at 127.18.168.131:56343
I20260810 18:48:35.246877 19106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:48:35.255800 23771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:35.248746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52866 (local address 127.18.168.190:41655)
0810 18:48:35.248868 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:35.248872 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:35.249106 (+ 234us) server_negotiation.cc:407] Connection header received
0810 18:48:35.249199 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:35.249201 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:35.249237 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:35.249309 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:35.250042 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:35.250520 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:35.251311 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:35.251512 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:35.251585 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:35.252824 (+ 1239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:35.252827 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:35.252828 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:35.252849 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:35.254273 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:35.254644 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:35.254646 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:35.254647 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:35.254767 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:35.255002 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:35.255004 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:35.255005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:35.255214 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:35.255320 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:35.255534 (+ 214us) server_negotiation.cc:299] Negotiation successful
0810 18:48:35.255622 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 18:48:35.258419 23718 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52866:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:48:35.261533 23718 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:48:35.276006 24276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:35.270146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55262 (local address 127.18.168.129:44433)
0810 18:48:35.270582 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:35.270590 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:35.270618 (+ 28us) server_negotiation.cc:407] Connection header received
0810 18:48:35.270695 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:35.270700 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:35.270924 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:35.271082 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:35.271606 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:35.272680 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:35.274346 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:35.275207 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:35.275329 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:35.275452 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:35.275729 (+ 277us) server_negotiation.cc:299] Negotiation successful
0810 18:48:35.275843 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":136,"threads_started":1}
I20260810 18:48:35.276006 24275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:35.270001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:44433 (local address 127.0.0.1:55262)
0810 18:48:35.270357 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:35.270408 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:35.270537 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:35.271216 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:35.271223 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:35.271253 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:35.271478 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:35.271491 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:35.272867 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:35.272869 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:35.274198 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:35.274208 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:35.275346 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:35.275351 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:35.275506 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:35.275553 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 18:48:35.275683 (+ 130us) client_negotiation.cc:240] Negotiation successful
0810 18:48:35.275805 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":105,"threads_started":1}
I20260810 18:48:35.276482 24277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:35.270825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.131:39285 (local address 127.0.0.1:46400)
0810 18:48:35.271351 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:35.271364 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:35.271476 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:35.272084 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:35.272087 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:35.272104 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:35.272199 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:35.272205 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:35.273778 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:35.273781 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:35.274883 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:35.274894 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:35.276079 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:35.276084 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:35.276205 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:35.276227 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 18:48:35.276292 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 18:48:35.276356 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":139,"threads_started":1}
I20260810 18:48:35.276949 24278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:35.270983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46400 (local address 127.18.168.131:39285)
0810 18:48:35.271375 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:35.271386 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:35.271441 (+ 55us) server_negotiation.cc:407] Connection header received
0810 18:48:35.271580 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:35.271586 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:35.271776 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:35.271938 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:35.272340 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:35.273631 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:35.275031 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:35.275774 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:35.275885 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:35.276007 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:35.276687 (+ 680us) server_negotiation.cc:299] Negotiation successful
0810 18:48:35.276801 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":93,"threads_started":1}
I20260810 18:48:35.277685 24279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:35.271250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:36635 (local address 127.0.0.1:40538)
0810 18:48:35.271747 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:35.271765 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:35.271868 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:35.272760 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:35.272763 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:35.272788 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:35.272888 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:35.272893 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:35.274773 (+ 1880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:35.274781 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:35.275849 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:35.275861 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:35.277249 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:35.277252 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:35.277353 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:35.277388 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 18:48:35.277461 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 18:48:35.277536 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"mutex_wait_us":1,"thread_start_us":103,"threads_started":1}
I20260810 18:48:35.277922 24280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:35.271700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40538 (local address 127.18.168.130:36635)
0810 18:48:35.272217 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:35.272227 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:35.272247 (+ 20us) server_negotiation.cc:407] Connection header received
0810 18:48:35.272374 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:35.272380 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:35.272588 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:35.272787 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:35.272985 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:35.274562 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:35.276036 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:35.277029 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:35.277254 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:35.277397 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:35.277631 (+ 234us) server_negotiation.cc:299] Negotiation successful
0810 18:48:35.277734 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":115,"threads_started":1}
I20260810 18:48:35.279352 23874 tablet_service.cc:1467] Processing CreateTablet for tablet 4648d3b9560540f2bf55d4e4954255a5 (DEFAULT_TABLE table=test-table [id=4bb99b54e6ea439fadf126d38bbac610]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:35.280359 23874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4648d3b9560540f2bf55d4e4954255a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:35.287328 24182 tablet_service.cc:1467] Processing CreateTablet for tablet 4648d3b9560540f2bf55d4e4954255a5 (DEFAULT_TABLE table=test-table [id=4bb99b54e6ea439fadf126d38bbac610]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:35.287971 24281 tablet_bootstrap.cc:492] T 4648d3b9560540f2bf55d4e4954255a5 P 7e5f2a8e152c4e45bd2f2a109bc6a8e9: Bootstrap starting.
I20260810 18:48:35.287590 24023 tablet_service.cc:1467] Processing CreateTablet for tablet 4648d3b9560540f2bf55d4e4954255a5 (DEFAULT_TABLE table=test-table [id=4bb99b54e6ea439fadf126d38bbac610]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:35.288511 24182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4648d3b9560540f2bf55d4e4954255a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:35.288628 24023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4648d3b9560540f2bf55d4e4954255a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:35.290585 24281 tablet_bootstrap.cc:654] T 4648d3b9560540f2bf55d4e4954255a5 P 7e5f2a8e152c4e45bd2f2a109bc6a8e9: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:35.292253 24281 log.cc:824] T 4648d3b9560540f2bf55d4e4954255a5 P 7e5f2a8e152c4e45bd2f2a109bc6a8e9: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:35.294577 24281 tablet_bootstrap.cc:492] T 4648d3b9560540f2bf55d4e4954255a5 P 7e5f2a8e152c4e45bd2f2a109bc6a8e9: No bootstrap required, opened a new log
I20260810 18:48:35.294853 24281 ts_tablet_manager.cc:1397] T 4648d3b9560540f2bf55d4e4954255a5 P 7e5f2a8e152c4e45bd2f2a109bc6a8e9: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.001s
I20260810 18:48:35.294896 24283 tablet_bootstrap.cc:492] T 4648d3b9560540f2bf55d4e4954255a5 P da7683da45a942cc9e4753af04f2b117: Bootstrap starting.
I20260810 18:48:35.296438 24284 tablet_bootstrap.cc:492] T 4648d3b9560540f2bf55d4e4954255a5 P 872d981c457f480faf66381c11d3097a: Bootstrap starting.
I20260810 18:48:35.297747 24283 tablet_bootstrap.cc:654] T 4648d3b9560540f2bf55d4e4954255a5 P da7683da45a942cc9e4753af04f2b117: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:35.298623 24283 log.cc:824] T 4648d3b9560540f2bf55d4e4954255a5 P da7683da45a942cc9e4753af04f2b117: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:35.298790 24284 tablet_bootstrap.cc:654] T 4648d3b9560540f2bf55d4e4954255a5 P 872d981c457f480faf66381c11d3097a: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:35.299443 24281 raft_consensus.cc:348] T 4648d3b9560540f2bf55d4e4954255a5 P 7e5f2a8e152c4e45bd2f2a109bc6a8e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e5f2a8e152c4e45bd2f2a109bc6a8e9" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44433 } } peers { permanent_uuid: "872d981c457f480faf66381c11d3097a" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 39285 } } peers { permanent_uuid: "da7683da45a942cc9e4753af04f2b117" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36635 } }
I20260810 18:48:35.299729 24281 raft_consensus.cc:374] T 4648d3b9560540f2bf55d4e4954255a5 P 7e5f2a8e152c4e45bd2f2a109bc6a8e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:35.299796 24281 raft_consensus.cc:729] T 4648d3b9560540f2bf55d4e4954255a5 P 7e5f2a8e152c4e45bd2f2a109bc6a8e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e5f2a8e152c4e45bd2f2a109bc6a8e9, State: Initialized, Role: FOLLOWER
I20260810 18:48:35.299937 24284 log.cc:824] T 4648d3b9560540f2bf55d4e4954255a5 P 872d981c457f480faf66381c11d3097a: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:35.300401 24281 consensus_queue.cc:260] T 4648d3b9560540f2bf55d4e4954255a5 P 7e5f2a8e152c4e45bd2f2a109bc6a8e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e5f2a8e152c4e45bd2f2a109bc6a8e9" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44433 } } peers { permanent_uuid: "872d981c457f480faf66381c11d3097a" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 39285 } } peers { permanent_uuid: "da7683da45a942cc9e4753af04f2b117" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36635 } }
I20260810 18:48:35.300669 24283 tablet_bootstrap.cc:492] T 4648d3b9560540f2bf55d4e4954255a5 P da7683da45a942cc9e4753af04f2b117: No bootstrap required, opened a new log
I20260810 18:48:35.300897 24283 ts_tablet_manager.cc:1397] T 4648d3b9560540f2bf55d4e4954255a5 P da7683da45a942cc9e4753af04f2b117: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260810 18:48:35.301448 24281 ts_tablet_manager.cc:1428] T 4648d3b9560540f2bf55d4e4954255a5 P 7e5f2a8e152c4e45bd2f2a109bc6a8e9: Time spent starting tablet: real 0.007s user 0.004s sys 0.003s
I20260810 18:48:35.301867 23950 heartbeater.cc:503] Master 127.18.168.190:41655 was elected leader, sending a full tablet report...
I20260810 18:48:35.302018 24284 tablet_bootstrap.cc:492] T 4648d3b9560540f2bf55d4e4954255a5 P 872d981c457f480faf66381c11d3097a: No bootstrap required, opened a new log
I20260810 18:48:35.302179 24284 ts_tablet_manager.cc:1397] T 4648d3b9560540f2bf55d4e4954255a5 P 872d981c457f480faf66381c11d3097a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 18:48:35.304706 24283 raft_consensus.cc:348] T 4648d3b9560540f2bf55d4e4954255a5 P da7683da45a942cc9e4753af04f2b117 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e5f2a8e152c4e45bd2f2a109bc6a8e9" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44433 } } peers { permanent_uuid: "872d981c457f480faf66381c11d3097a" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 39285 } } peers { permanent_uuid: "da7683da45a942cc9e4753af04f2b117" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36635 } }
I20260810 18:48:35.304970 24283 raft_consensus.cc:374] T 4648d3b9560540f2bf55d4e4954255a5 P da7683da45a942cc9e4753af04f2b117 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:35.305025 24283 raft_consensus.cc:729] T 4648d3b9560540f2bf55d4e4954255a5 P da7683da45a942cc9e4753af04f2b117 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da7683da45a942cc9e4753af04f2b117, State: Initialized, Role: FOLLOWER
I20260810 18:48:35.305473 24283 consensus_queue.cc:260] T 4648d3b9560540f2bf55d4e4954255a5 P da7683da45a942cc9e4753af04f2b117 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e5f2a8e152c4e45bd2f2a109bc6a8e9" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44433 } } peers { permanent_uuid: "872d981c457f480faf66381c11d3097a" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 39285 } } peers { permanent_uuid: "da7683da45a942cc9e4753af04f2b117" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36635 } }
I20260810 18:48:35.305871 24284 raft_consensus.cc:348] T 4648d3b9560540f2bf55d4e4954255a5 P 872d981c457f480faf66381c11d3097a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e5f2a8e152c4e45bd2f2a109bc6a8e9" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44433 } } peers { permanent_uuid: "872d981c457f480faf66381c11d3097a" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 39285 } } peers { permanent_uuid: "da7683da45a942cc9e4753af04f2b117" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36635 } }
I20260810 18:48:35.306113 24284 raft_consensus.cc:374] T 4648d3b9560540f2bf55d4e4954255a5 P 872d981c457f480faf66381c11d3097a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:35.306167 24284 raft_consensus.cc:729] T 4648d3b9560540f2bf55d4e4954255a5 P 872d981c457f480faf66381c11d3097a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 872d981c457f480faf66381c11d3097a, State: Initialized, Role: FOLLOWER
I20260810 18:48:35.306284 24101 heartbeater.cc:503] Master 127.18.168.190:41655 was elected leader, sending a full tablet report...
I20260810 18:48:35.306459 24283 ts_tablet_manager.cc:1428] T 4648d3b9560540f2bf55d4e4954255a5 P da7683da45a942cc9e4753af04f2b117: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20260810 18:48:35.306566 24284 consensus_queue.cc:260] T 4648d3b9560540f2bf55d4e4954255a5 P 872d981c457f480faf66381c11d3097a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e5f2a8e152c4e45bd2f2a109bc6a8e9" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44433 } } peers { permanent_uuid: "872d981c457f480faf66381c11d3097a" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 39285 } } peers { permanent_uuid: "da7683da45a942cc9e4753af04f2b117" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36635 } }
I20260810 18:48:35.307219 24263 heartbeater.cc:503] Master 127.18.168.190:41655 was elected leader, sending a full tablet report...
I20260810 18:48:35.307541 24284 ts_tablet_manager.cc:1428] T 4648d3b9560540f2bf55d4e4954255a5 P 872d981c457f480faf66381c11d3097a: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260810 18:48:35.475739 24264 tablet.cc:2369] T 4648d3b9560540f2bf55d4e4954255a5 P 872d981c457f480faf66381c11d3097a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:48:35.499900 24102 tablet.cc:2369] T 4648d3b9560540f2bf55d4e4954255a5 P da7683da45a942cc9e4753af04f2b117: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:48:35.520159 24289 raft_consensus.cc:482] T 4648d3b9560540f2bf55d4e4954255a5 P 872d981c457f480faf66381c11d3097a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:48:35.520370 24289 raft_consensus.cc:504] T 4648d3b9560540f2bf55d4e4954255a5 P 872d981c457f480faf66381c11d3097a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e5f2a8e152c4e45bd2f2a109bc6a8e9" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44433 } } peers { permanent_uuid: "872d981c457f480faf66381c11d3097a" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 39285 } } peers { permanent_uuid: "da7683da45a942cc9e4753af04f2b117" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36635 } }
I20260810 18:48:35.521629 24289 leader_election.cc:288] T 4648d3b9560540f2bf55d4e4954255a5 P 872d981c457f480faf66381c11d3097a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e5f2a8e152c4e45bd2f2a109bc6a8e9 (127.18.168.129:44433), da7683da45a942cc9e4753af04f2b117 (127.18.168.130:36635)
I20260810 18:48:35.525695 24265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:35.521856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:44433 (local address 127.18.168.131:36799)
0810 18:48:35.522015 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:35.522064 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:35.522182 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:35.522577 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:35.522580 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:35.522596 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:35.522688 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:35.522693 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:35.523607 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:35.523610 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:35.524523 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:35.524533 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:35.525286 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:35.525290 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:35.525396 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:35.525424 (+ 28us) client_negotiation.cc:769] Sending connection context
0810 18:48:35.525502 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 18:48:35.525572 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260810 18:48:35.525738 24276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:35.522001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:36799 (local address 127.18.168.129:44433)
0810 18:48:35.522155 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:35.522160 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:35.522176 (+ 16us) server_negotiation.cc:407] Connection header received
0810 18:48:35.522310 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:35.522315 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:35.522370 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:35.522472 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:35.522797 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:35.523489 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:35.524652 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:35.525172 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:35.525264 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:35.525304 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:35.525534 (+ 230us) server_negotiation.cc:299] Negotiation successful
0810 18:48:35.525608 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 18:48:35.525952 24290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:35.522161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:36635 (local address 127.18.168.131:53159)
0810 18:48:35.522578 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:35.522588 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:35.522700 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:35.522990 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:35.522993 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:35.523012 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:35.523127 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:35.523133 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:35.523966 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:35.523969 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:35.524830 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:35.524840 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:35.525551 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:35.525555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:35.525650 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:35.525672 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 18:48:35.525752 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 18:48:35.525823 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":118,"threads_started":1}
I20260810 18:48:35.526012 24280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:35.522311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:53159 (local address 127.18.168.130:36635)
0810 18:48:35.522454 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:35.522459 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:35.522663 (+ 204us) server_negotiation.cc:407] Connection header received
0810 18:48:35.522850 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:35.522853 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:35.522893 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:35.522948 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:35.523234 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:35.523850 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:35.524945 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:35.525401 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:35.525527 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:35.525571 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:35.525787 (+ 216us) server_negotiation.cc:299] Negotiation successful
0810 18:48:35.525870 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 18:48:35.526356 23897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4648d3b9560540f2bf55d4e4954255a5" candidate_uuid: "872d981c457f480faf66381c11d3097a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e5f2a8e152c4e45bd2f2a109bc6a8e9" is_pre_election: true
I20260810 18:48:35.526687 23897 raft_consensus.cc:2454] T 4648d3b9560540f2bf55d4e4954255a5 P 7e5f2a8e152c4e45bd2f2a109bc6a8e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 872d981c457f480faf66381c11d3097a in term 0.
I20260810 18:48:35.526703 24044 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4648d3b9560540f2bf55d4e4954255a5" candidate_uuid: "872d981c457f480faf66381c11d3097a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da7683da45a942cc9e4753af04f2b117" is_pre_election: true
I20260810 18:48:35.526952 24044 raft_consensus.cc:2454] T 4648d3b9560540f2bf55d4e4954255a5 P da7683da45a942cc9e4753af04f2b117 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 872d981c457f480faf66381c11d3097a in term 0.
I20260810 18:48:35.527226 24126 leader_election.cc:302] T 4648d3b9560540f2bf55d4e4954255a5 P 872d981c457f480faf66381c11d3097a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e5f2a8e152c4e45bd2f2a109bc6a8e9, 872d981c457f480faf66381c11d3097a; no voters:
I20260810 18:48:35.527526 24289 raft_consensus.cc:2781] T 4648d3b9560540f2bf55d4e4954255a5 P 872d981c457f480faf66381c11d3097a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:48:35.527606 24289 raft_consensus.cc:482] T 4648d3b9560540f2bf55d4e4954255a5 P 872d981c457f480faf66381c11d3097a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:48:35.527637 24289 raft_consensus.cc:3037] T 4648d3b9560540f2bf55d4e4954255a5 P 872d981c457f480faf66381c11d3097a [term 0 FOLLOWER]: Advancing to term 1
W20260810 18:48:35.527827 23951 tablet.cc:2369] T 4648d3b9560540f2bf55d4e4954255a5 P 7e5f2a8e152c4e45bd2f2a109bc6a8e9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:48:35.528818 24289 raft_consensus.cc:504] T 4648d3b9560540f2bf55d4e4954255a5 P 872d981c457f480faf66381c11d3097a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e5f2a8e152c4e45bd2f2a109bc6a8e9" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44433 } } peers { permanent_uuid: "872d981c457f480faf66381c11d3097a" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 39285 } } peers { permanent_uuid: "da7683da45a942cc9e4753af04f2b117" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36635 } }
I20260810 18:48:35.529148 24289 leader_election.cc:288] T 4648d3b9560540f2bf55d4e4954255a5 P 872d981c457f480faf66381c11d3097a [CANDIDATE]: Term 1 election: Requested vote from peers 7e5f2a8e152c4e45bd2f2a109bc6a8e9 (127.18.168.129:44433), da7683da45a942cc9e4753af04f2b117 (127.18.168.130:36635)
I20260810 18:48:35.529585 24044 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4648d3b9560540f2bf55d4e4954255a5" candidate_uuid: "872d981c457f480faf66381c11d3097a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da7683da45a942cc9e4753af04f2b117"
I20260810 18:48:35.529594 23897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4648d3b9560540f2bf55d4e4954255a5" candidate_uuid: "872d981c457f480faf66381c11d3097a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e5f2a8e152c4e45bd2f2a109bc6a8e9"
I20260810 18:48:35.529697 23897 raft_consensus.cc:3037] T 4648d3b9560540f2bf55d4e4954255a5 P 7e5f2a8e152c4e45bd2f2a109bc6a8e9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:35.529734 24044 raft_consensus.cc:3037] T 4648d3b9560540f2bf55d4e4954255a5 P da7683da45a942cc9e4753af04f2b117 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:35.530737 23897 raft_consensus.cc:2454] T 4648d3b9560540f2bf55d4e4954255a5 P 7e5f2a8e152c4e45bd2f2a109bc6a8e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 872d981c457f480faf66381c11d3097a in term 1.
I20260810 18:48:35.530814 24044 raft_consensus.cc:2454] T 4648d3b9560540f2bf55d4e4954255a5 P da7683da45a942cc9e4753af04f2b117 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 872d981c457f480faf66381c11d3097a in term 1.
I20260810 18:48:35.531155 24126 leader_election.cc:302] T 4648d3b9560540f2bf55d4e4954255a5 P 872d981c457f480faf66381c11d3097a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 872d981c457f480faf66381c11d3097a, da7683da45a942cc9e4753af04f2b117; no voters:
I20260810 18:48:35.531380 24289 raft_consensus.cc:2781] T 4648d3b9560540f2bf55d4e4954255a5 P 872d981c457f480faf66381c11d3097a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:35.531622 24289 raft_consensus.cc:686] T 4648d3b9560540f2bf55d4e4954255a5 P 872d981c457f480faf66381c11d3097a [term 1 LEADER]: Becoming Leader. State: Replica: 872d981c457f480faf66381c11d3097a, State: Running, Role: LEADER
I20260810 18:48:35.531956 24289 consensus_queue.cc:237] T 4648d3b9560540f2bf55d4e4954255a5 P 872d981c457f480faf66381c11d3097a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e5f2a8e152c4e45bd2f2a109bc6a8e9" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44433 } } peers { permanent_uuid: "872d981c457f480faf66381c11d3097a" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 39285 } } peers { permanent_uuid: "da7683da45a942cc9e4753af04f2b117" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36635 } }
I20260810 18:48:35.535080 23716 catalog_manager.cc:5495] T 4648d3b9560540f2bf55d4e4954255a5 P 872d981c457f480faf66381c11d3097a reported cstate change: term changed from 0 to 1, leader changed from <none> to 872d981c457f480faf66381c11d3097a (127.18.168.131). New cstate: current_term: 1 leader_uuid: "872d981c457f480faf66381c11d3097a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e5f2a8e152c4e45bd2f2a109bc6a8e9" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "872d981c457f480faf66381c11d3097a" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 39285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da7683da45a942cc9e4753af04f2b117" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36635 } health_report { overall_health: UNKNOWN } } }
I20260810 18:48:35.565982 24278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:35.562768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46416 (local address 127.18.168.131:39285)
0810 18:48:35.562876 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:35.562880 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:35.562893 (+ 13us) server_negotiation.cc:407] Connection header received
0810 18:48:35.563006 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:35.563008 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:35.563059 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:35.563142 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:48:35.563355 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:35.563828 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:35.564640 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:35.564907 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:35.564988 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:35.565271 (+ 283us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:48:35.565684 (+ 413us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:48:35.565770 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:35.565823 (+ 53us) server_negotiation.cc:299] Negotiation successful
0810 18:48:35.565867 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 18:48:35.571704 24044 raft_consensus.cc:1261] T 4648d3b9560540f2bf55d4e4954255a5 P da7683da45a942cc9e4753af04f2b117 [term 1 FOLLOWER]: Refusing update from remote peer 872d981c457f480faf66381c11d3097a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:35.571768 23897 raft_consensus.cc:1261] T 4648d3b9560540f2bf55d4e4954255a5 P 7e5f2a8e152c4e45bd2f2a109bc6a8e9 [term 1 FOLLOWER]: Refusing update from remote peer 872d981c457f480faf66381c11d3097a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:35.572429 24289 consensus_queue.cc:1035] T 4648d3b9560540f2bf55d4e4954255a5 P 872d981c457f480faf66381c11d3097a [LEADER]: Connected to new peer: Peer: permanent_uuid: "da7683da45a942cc9e4753af04f2b117" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:35.572580 24291 consensus_queue.cc:1035] T 4648d3b9560540f2bf55d4e4954255a5 P 872d981c457f480faf66381c11d3097a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e5f2a8e152c4e45bd2f2a109bc6a8e9" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:35.583290 24297 mvcc.cc:205] Tried to move back new op lower bound from 7317044082974154752 to 7317044082821443584. Current Snapshot: MvccSnapshot[applied={T|T < 7317044082974154752}]
I20260810 18:48:35.587710 23718 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:35.587885 23718 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:35.590305 23718 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c2305433a4634510925ef775003a48bd: Sending DeleteTablet for 3 replicas of tablet 4648d3b9560540f2bf55d4e4954255a5
I20260810 18:48:35.591015 23874 tablet_service.cc:1514] Processing DeleteTablet for tablet 4648d3b9560540f2bf55d4e4954255a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:35 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:55262
I20260810 18:48:35.591185 24182 tablet_service.cc:1514] Processing DeleteTablet for tablet 4648d3b9560540f2bf55d4e4954255a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:35 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:46400
I20260810 18:48:35.591522 24023 tablet_service.cc:1514] Processing DeleteTablet for tablet 4648d3b9560540f2bf55d4e4954255a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:35 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:40538
I20260810 18:48:35.591666 24306 tablet_replica.cc:330] T 4648d3b9560540f2bf55d4e4954255a5 P 872d981c457f480faf66381c11d3097a: stopping tablet replica
I20260810 18:48:35.591987 24306 raft_consensus.cc:2229] T 4648d3b9560540f2bf55d4e4954255a5 P 872d981c457f480faf66381c11d3097a [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:48:35.592199 24305 tablet_replica.cc:330] T 4648d3b9560540f2bf55d4e4954255a5 P 7e5f2a8e152c4e45bd2f2a109bc6a8e9: stopping tablet replica
I20260810 18:48:35.592324 24306 raft_consensus.cc:2258] T 4648d3b9560540f2bf55d4e4954255a5 P 872d981c457f480faf66381c11d3097a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:35.592495 24305 raft_consensus.cc:2229] T 4648d3b9560540f2bf55d4e4954255a5 P 7e5f2a8e152c4e45bd2f2a109bc6a8e9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:35.592844 24305 raft_consensus.cc:2258] T 4648d3b9560540f2bf55d4e4954255a5 P 7e5f2a8e152c4e45bd2f2a109bc6a8e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:35.592864 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 23803
I20260810 18:48:35.592003 24307 tablet_replica.cc:330] T 4648d3b9560540f2bf55d4e4954255a5 P da7683da45a942cc9e4753af04f2b117: stopping tablet replica
I20260810 18:48:35.602010 24306 ts_tablet_manager.cc:1905] T 4648d3b9560540f2bf55d4e4954255a5 P 872d981c457f480faf66381c11d3097a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:35.602347 24307 raft_consensus.cc:2229] T 4648d3b9560540f2bf55d4e4954255a5 P da7683da45a942cc9e4753af04f2b117 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:35.602710 24307 raft_consensus.cc:2258] T 4648d3b9560540f2bf55d4e4954255a5 P da7683da45a942cc9e4753af04f2b117 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:35.604138 24307 ts_tablet_manager.cc:1905] T 4648d3b9560540f2bf55d4e4954255a5 P da7683da45a942cc9e4753af04f2b117: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:35.605581 24306 ts_tablet_manager.cc:1918] T 4648d3b9560540f2bf55d4e4954255a5 P 872d981c457f480faf66381c11d3097a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:48:35.605685 24306 log.cc:1196] T 4648d3b9560540f2bf55d4e4954255a5 P 872d981c457f480faf66381c11d3097a: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-2/wal/wals/4648d3b9560540f2bf55d4e4954255a5
I20260810 18:48:35.606024 24306 ts_tablet_manager.cc:1939] T 4648d3b9560540f2bf55d4e4954255a5 P 872d981c457f480faf66381c11d3097a: Deleting consensus metadata
I20260810 18:48:35.607121 24307 ts_tablet_manager.cc:1918] T 4648d3b9560540f2bf55d4e4954255a5 P da7683da45a942cc9e4753af04f2b117: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:48:35.607278 24307 log.cc:1196] T 4648d3b9560540f2bf55d4e4954255a5 P da7683da45a942cc9e4753af04f2b117: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387687857264-19106-0/minicluster-data/ts-1/wal/wals/4648d3b9560540f2bf55d4e4954255a5
I20260810 18:48:35.607266 23703 catalog_manager.cc:4841] TS 872d981c457f480faf66381c11d3097a (127.18.168.131:39285): tablet 4648d3b9560540f2bf55d4e4954255a5 (table test-table [id=4bb99b54e6ea439fadf126d38bbac610]) successfully deleted
I20260810 18:48:35.607594 24307 ts_tablet_manager.cc:1939] T 4648d3b9560540f2bf55d4e4954255a5 P da7683da45a942cc9e4753af04f2b117: Deleting consensus metadata
I20260810 18:48:35.608747 23703 catalog_manager.cc:4841] TS da7683da45a942cc9e4753af04f2b117 (127.18.168.130:36635): tablet 4648d3b9560540f2bf55d4e4954255a5 (table test-table [id=4bb99b54e6ea439fadf126d38bbac610]) successfully deleted
W20260810 18:48:35.609442 23703 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.168.129:44433 (error 108)
I20260810 18:48:35.609561 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 23956
I20260810 18:48:35.612254 24279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:35.611917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:44433 (local address 127.0.0.1:55274)
0810 18:48:35.612068 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:35.612158 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.129:44433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260810 18:48:35.612484 23703 catalog_manager.cc:4584] TS 7e5f2a8e152c4e45bd2f2a109bc6a8e9 (127.18.168.129:44433): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4648d3b9560540f2bf55d4e4954255a5: Network error: Client connection negotiation failed: client connection to 127.18.168.129:44433: connect: Connection refused (error 111)
I20260810 18:48:35.618654 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 24106
I20260810 18:48:35.626076 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 23680
2026-08-10T18:48:35Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2558 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-10T18:48:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:48:35Z Disabled control of system clock
I20260810 18:48:35.658771 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:43563
--webserver_interface=127.18.168.190
--webserver_port=0
--builtin_ntp_servers=127.18.168.148:46165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:43563
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 18:48:35.767684 24314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:35.767925 24314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:35.767987 24314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:35.772390 24314 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:48:35.772464 24314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:35.772488 24314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:35.772519 24314 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:48:35.772559 24314 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:48:35.777194 24314 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:46165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:43563
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:43563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24314
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:35.778216 24314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:35.779556 24314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:35.788129 24321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:35.788198 24314 server_base.cc:1034] running on GCE node
W20260810 18:48:35.788144 24323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:35.788127 24319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:35.788915 24314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:35.789923 24314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:35.791083 24314 hybrid_clock.cc:648] HybridClock initialized: now 1786387715791065 us; error 29 us; skew 500 ppm
I20260810 18:48:35.793135 24314 webserver.cc:460] Webserver started at http://127.18.168.190:45845/ using document root <none> and password file <none>
I20260810 18:48:35.793741 24314 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:35.793798 24314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:35.793999 24314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:35.795972 24314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "94788d10774c4ccc8e74a6ecc8778206"
format_stamp: "Formatted at 2026-08-10 18:48:35 on dist-test-slave-cbsf"
server_key: "d662b3b67f071ac67fe377af47dd82ad"
server_key_iv: "6186e96fd7c1147ce424a04201c2ccbf"
server_key_version: "encryptionkey@0"
I20260810 18:48:35.796538 24314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "94788d10774c4ccc8e74a6ecc8778206"
format_stamp: "Formatted at 2026-08-10 18:48:35 on dist-test-slave-cbsf"
server_key: "d662b3b67f071ac67fe377af47dd82ad"
server_key_iv: "6186e96fd7c1147ce424a04201c2ccbf"
server_key_version: "encryptionkey@0"
I20260810 18:48:35.800139 24314 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:48:35.802539 24329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:35.803776 24314 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:48:35.803907 24314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "94788d10774c4ccc8e74a6ecc8778206"
format_stamp: "Formatted at 2026-08-10 18:48:35 on dist-test-slave-cbsf"
server_key: "d662b3b67f071ac67fe377af47dd82ad"
server_key_iv: "6186e96fd7c1147ce424a04201c2ccbf"
server_key_version: "encryptionkey@0"
I20260810 18:48:35.804013 24314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:35.808511 24314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:35.809149 24314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:35.809332 24314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:35.816043 24314 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:43563
I20260810 18:48:35.816049 24390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:43563 every 8 connection(s)
I20260810 18:48:35.816998 24314 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:48:35.820143 24391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:35.824120 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 24314
I20260810 18:48:35.824278 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:48:35.824542 19106 external_mini_cluster.cc:1428] Setting key fc48999c552d30ec55c95d856df7a887
I20260810 18:48:35.825716 24391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94788d10774c4ccc8e74a6ecc8778206: Bootstrap starting.
I20260810 18:48:35.827800 24391 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94788d10774c4ccc8e74a6ecc8778206: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:35.828779 24391 log.cc:824] T 00000000000000000000000000000000 P 94788d10774c4ccc8e74a6ecc8778206: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:35.829962 24394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:35.825963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46648 (local address 127.18.168.190:43563)
0810 18:48:35.826559 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:35.826574 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:35.826607 (+ 33us) server_negotiation.cc:407] Connection header received
0810 18:48:35.827208 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:35.827222 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:35.827544 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:35.827814 (+ 270us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:35.828742 (+ 928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:35.828747 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:35.828758 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:48:35.828769 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:35.828824 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:48:35.828854 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:35.828974 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:35.829080 (+ 106us) server_negotiation.cc:299] Negotiation successful
0810 18:48:35.829323 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":122,"threads_started":1}
I20260810 18:48:35.830690 24391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94788d10774c4ccc8e74a6ecc8778206: No bootstrap required, opened a new log
I20260810 18:48:35.834556 24391 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94788d10774c4ccc8e74a6ecc8778206 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94788d10774c4ccc8e74a6ecc8778206" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43563 } }
I20260810 18:48:35.834838 24391 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94788d10774c4ccc8e74a6ecc8778206 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:35.834900 24391 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94788d10774c4ccc8e74a6ecc8778206 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94788d10774c4ccc8e74a6ecc8778206, State: Initialized, Role: FOLLOWER
I20260810 18:48:35.835326 24391 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94788d10774c4ccc8e74a6ecc8778206 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94788d10774c4ccc8e74a6ecc8778206" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43563 } }
I20260810 18:48:35.835446 24391 raft_consensus.cc:388] T 00000000000000000000000000000000 P 94788d10774c4ccc8e74a6ecc8778206 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:48:35.835497 24391 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94788d10774c4ccc8e74a6ecc8778206 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:48:35.835567 24391 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94788d10774c4ccc8e74a6ecc8778206 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:35.836424 24391 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94788d10774c4ccc8e74a6ecc8778206 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94788d10774c4ccc8e74a6ecc8778206" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43563 } }
I20260810 18:48:35.836763 24391 leader_election.cc:302] T 00000000000000000000000000000000 P 94788d10774c4ccc8e74a6ecc8778206 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94788d10774c4ccc8e74a6ecc8778206; no voters:
I20260810 18:48:35.837010 24391 leader_election.cc:288] T 00000000000000000000000000000000 P 94788d10774c4ccc8e74a6ecc8778206 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:48:35.837149 24396 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94788d10774c4ccc8e74a6ecc8778206 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:35.837440 24396 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94788d10774c4ccc8e74a6ecc8778206 [term 1 LEADER]: Becoming Leader. State: Replica: 94788d10774c4ccc8e74a6ecc8778206, State: Running, Role: LEADER
I20260810 18:48:35.837724 24396 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94788d10774c4ccc8e74a6ecc8778206 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94788d10774c4ccc8e74a6ecc8778206" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43563 } }
I20260810 18:48:35.838295 24391 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94788d10774c4ccc8e74a6ecc8778206 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:48:35.839371 24398 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94788d10774c4ccc8e74a6ecc8778206 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94788d10774c4ccc8e74a6ecc8778206. Latest consensus state: current_term: 1 leader_uuid: "94788d10774c4ccc8e74a6ecc8778206" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94788d10774c4ccc8e74a6ecc8778206" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43563 } } }
I20260810 18:48:35.839516 24398 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94788d10774c4ccc8e74a6ecc8778206 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:35.839810 24397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94788d10774c4ccc8e74a6ecc8778206 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "94788d10774c4ccc8e74a6ecc8778206" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94788d10774c4ccc8e74a6ecc8778206" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43563 } } }
I20260810 18:48:35.839900 24397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94788d10774c4ccc8e74a6ecc8778206 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:35.839993 24405 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:48:35.842859 24405 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:48:35.848021 24405 catalog_manager.cc:1347] Generated new cluster ID: 4b65d23a42ce43f988f2a6945ff1eb3a
I20260810 18:48:35.848093 24405 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:48:35.857398 24405 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:48:35.857705 24405 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:48:35.862612 24405 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 94788d10774c4ccc8e74a6ecc8778206: Generated new TSK 0
I20260810 18:48:35.863174 24405 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:48:35.871966 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.129:0
--local_ip_for_outbound_sockets=127.18.168.129
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:43563
--builtin_ntp_servers=127.18.168.148:46165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 18:48:35.985071 24416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:35.985337 24416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:35.985383 24416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:35.989593 24416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:35.989638 24416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:35.989694 24416 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.129
I20260810 18:48:35.994369 24416 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:46165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:43563
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.129
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:35.995317 24416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:35.996618 24416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:36.006212 24425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:36.006237 24421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:36.006212 24422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:36.006500 24416 server_base.cc:1034] running on GCE node
I20260810 18:48:36.006940 24416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:36.007663 24416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:36.008831 24416 hybrid_clock.cc:648] HybridClock initialized: now 1786387716008808 us; error 49 us; skew 500 ppm
I20260810 18:48:36.010578 24416 webserver.cc:460] Webserver started at http://127.18.168.129:44273/ using document root <none> and password file <none>
I20260810 18:48:36.011148 24416 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:36.011240 24416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:36.011466 24416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:36.013557 24416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-0/data/instance:
uuid: "37a52916ede44230b681def2312027df"
format_stamp: "Formatted at 2026-08-10 18:48:36 on dist-test-slave-cbsf"
server_key: "d16bb382d453a9fc6fc5cc91aabf5655"
server_key_iv: "4dc90a470330bee80d4fd600b8863c49"
server_key_version: "encryptionkey@0"
I20260810 18:48:36.014109 24416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance:
uuid: "37a52916ede44230b681def2312027df"
format_stamp: "Formatted at 2026-08-10 18:48:36 on dist-test-slave-cbsf"
server_key: "d16bb382d453a9fc6fc5cc91aabf5655"
server_key_iv: "4dc90a470330bee80d4fd600b8863c49"
server_key_version: "encryptionkey@0"
I20260810 18:48:36.017828 24416 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:48:36.020190 24431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:36.021420 24416 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:48:36.021570 24416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-0/wal
uuid: "37a52916ede44230b681def2312027df"
format_stamp: "Formatted at 2026-08-10 18:48:36 on dist-test-slave-cbsf"
server_key: "d16bb382d453a9fc6fc5cc91aabf5655"
server_key_iv: "4dc90a470330bee80d4fd600b8863c49"
server_key_version: "encryptionkey@0"
I20260810 18:48:36.021682 24416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:36.025980 24416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:36.026548 24416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:36.026726 24416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:36.027251 24416 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:36.028170 24416 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:36.028242 24416 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:36.028308 24416 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:36.028347 24416 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:36.036906 24416 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.129:44431
I20260810 18:48:36.036972 24562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.129:44431 every 8 connection(s)
I20260810 18:48:36.037817 24416 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
I20260810 18:48:36.038825 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 24416
I20260810 18:48:36.038961 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance
I20260810 18:48:36.039248 19106 external_mini_cluster.cc:1428] Setting key fb4199a8fe7983d645efe6bb80957c7f
I20260810 18:48:36.041407 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.130:0
--local_ip_for_outbound_sockets=127.18.168.130
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:43563
--builtin_ntp_servers=127.18.168.148:46165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260810 18:48:36.044541 24394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:36.039813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:44595 (local address 127.18.168.190:43563)
0810 18:48:36.039952 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:36.039955 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:36.041209 (+ 1254us) server_negotiation.cc:407] Connection header received
0810 18:48:36.042728 (+ 1519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:36.042731 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:36.042776 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:36.042856 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:36.044056 (+ 1200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:36.044062 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:36.044066 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:48:36.044081 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:36.044108 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:48:36.044118 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:36.044237 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:36.044331 (+ 94us) server_negotiation.cc:299] Negotiation successful
0810 18:48:36.044379 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 18:48:36.045373 24566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:36.040276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43563 (local address 127.18.168.129:44595)
0810 18:48:36.041059 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:36.041100 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:36.042506 (+ 1406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:36.043027 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:36.043034 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:36.043120 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:36.043775 (+ 655us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:48:36.043777 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:36.043819 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:48:36.043888 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:36.044220 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:36.044227 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:36.044232 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 18:48:36.044361 (+ 129us) client_negotiation.cc:240] Negotiation successful
0810 18:48:36.044559 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":161,"threads_started":1}
I20260810 18:48:36.046639 24564 heartbeater.cc:347] Connected to a master server at 127.18.168.190:43563
I20260810 18:48:36.046886 24564 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:36.047394 24564 heartbeater.cc:511] Master 127.18.168.190:43563 requested a full tablet report, sending...
I20260810 18:48:36.049158 24351 ts_manager.cc:194] Registered new tserver with Master: 37a52916ede44230b681def2312027df (127.18.168.129:44431)
W20260810 18:48:36.157944 24567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:36.158176 24567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:36.158215 24567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:36.162246 24567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:36.162295 24567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:36.162382 24567 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.130
I20260810 18:48:36.167122 24567 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:46165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:43563
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.130
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:36.168144 24567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:36.169540 24567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:36.178797 24574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:36.178773 24575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:36.179104 24567 server_base.cc:1034] running on GCE node
W20260810 18:48:36.178768 24577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:36.179524 24567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:36.180095 24567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:36.181247 24567 hybrid_clock.cc:648] HybridClock initialized: now 1786387716181217 us; error 40 us; skew 500 ppm
I20260810 18:48:36.183024 24567 webserver.cc:460] Webserver started at http://127.18.168.130:46703/ using document root <none> and password file <none>
I20260810 18:48:36.183593 24567 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:36.183655 24567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:36.183904 24567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:36.185992 24567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-1/data/instance:
uuid: "4ec20fce80b5414185b09bd097d8f57e"
format_stamp: "Formatted at 2026-08-10 18:48:36 on dist-test-slave-cbsf"
server_key: "d8a5fbc637eceb40f12e8420590ce14f"
server_key_iv: "99b95e46bae3059920cd295d997f74ff"
server_key_version: "encryptionkey@0"
I20260810 18:48:36.186558 24567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance:
uuid: "4ec20fce80b5414185b09bd097d8f57e"
format_stamp: "Formatted at 2026-08-10 18:48:36 on dist-test-slave-cbsf"
server_key: "d8a5fbc637eceb40f12e8420590ce14f"
server_key_iv: "99b95e46bae3059920cd295d997f74ff"
server_key_version: "encryptionkey@0"
I20260810 18:48:36.190140 24567 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:48:36.192466 24583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:36.193724 24567 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:48:36.193862 24567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-1/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-1/wal
uuid: "4ec20fce80b5414185b09bd097d8f57e"
format_stamp: "Formatted at 2026-08-10 18:48:36 on dist-test-slave-cbsf"
server_key: "d8a5fbc637eceb40f12e8420590ce14f"
server_key_iv: "99b95e46bae3059920cd295d997f74ff"
server_key_version: "encryptionkey@0"
I20260810 18:48:36.193969 24567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:36.198498 24567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:36.199090 24567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:36.199266 24567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:36.199869 24567 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:36.200826 24567 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:36.200892 24567 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:36.200955 24567 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:36.201005 24567 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:36.210076 24567 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.130:36163
I20260810 18:48:36.210099 24716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.130:36163 every 8 connection(s)
I20260810 18:48:36.210955 24567 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
I20260810 18:48:36.217119 24394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:36.212798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:50165 (local address 127.18.168.190:43563)
0810 18:48:36.212932 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:36.212936 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:36.213920 (+ 984us) server_negotiation.cc:407] Connection header received
0810 18:48:36.215226 (+ 1306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:36.215229 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:36.215287 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:36.215396 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:36.216604 (+ 1208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:36.216607 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:36.216610 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:48:36.216622 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:36.216644 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:48:36.216659 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:36.216793 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:36.216937 (+ 144us) server_negotiation.cc:299] Negotiation successful
0810 18:48:36.216982 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 18:48:36.217905 24719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:36.213161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43563 (local address 127.18.168.130:50165)
0810 18:48:36.213783 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:36.213820 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:36.214987 (+ 1167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:36.215615 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:36.215624 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:36.215725 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:36.216336 (+ 611us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:48:36.216338 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:36.216378 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:48:36.216444 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:36.216774 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:36.216783 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:36.216789 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 18:48:36.216977 (+ 188us) client_negotiation.cc:240] Negotiation successful
0810 18:48:36.217221 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":114,"threads_started":1}
I20260810 18:48:36.219038 24717 heartbeater.cc:347] Connected to a master server at 127.18.168.190:43563
I20260810 18:48:36.219280 24717 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:36.219676 24717 heartbeater.cc:511] Master 127.18.168.190:43563 requested a full tablet report, sending...
I20260810 18:48:36.220057 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 24567
I20260810 18:48:36.220201 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance
I20260810 18:48:36.220477 19106 external_mini_cluster.cc:1428] Setting key f28fd1ec1dc6c16adb04ae0a7326cb65
I20260810 18:48:36.220755 24351 ts_manager.cc:194] Registered new tserver with Master: 4ec20fce80b5414185b09bd097d8f57e (127.18.168.130:36163)
I20260810 18:48:36.222366 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.131:0
--local_ip_for_outbound_sockets=127.18.168.131
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:43563
--builtin_ntp_servers=127.18.168.148:46165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 18:48:36.334642 24721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:36.334873 24721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:36.334942 24721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:36.339028 24721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:36.339077 24721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:36.339166 24721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.131
I20260810 18:48:36.343927 24721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:46165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:43563
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.131
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:36.345027 24721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:36.346385 24721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:36.356413 24727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:36.356449 24721 server_base.cc:1034] running on GCE node
W20260810 18:48:36.356410 24728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:36.356426 24731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:36.357177 24721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:36.357807 24721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:36.359000 24721 hybrid_clock.cc:648] HybridClock initialized: now 1786387716358981 us; error 29 us; skew 500 ppm
I20260810 18:48:36.360844 24721 webserver.cc:460] Webserver started at http://127.18.168.131:32801/ using document root <none> and password file <none>
I20260810 18:48:36.361428 24721 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:36.361523 24721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:36.361732 24721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:36.363806 24721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-2/data/instance:
uuid: "0519acbb10bf4777b6184374d4994e1a"
format_stamp: "Formatted at 2026-08-10 18:48:36 on dist-test-slave-cbsf"
server_key: "cad116c898564637905c7d23a3f5996b"
server_key_iv: "cb1e14412da8ba384ed12c1aa4e789b9"
server_key_version: "encryptionkey@0"
I20260810 18:48:36.364372 24721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance:
uuid: "0519acbb10bf4777b6184374d4994e1a"
format_stamp: "Formatted at 2026-08-10 18:48:36 on dist-test-slave-cbsf"
server_key: "cad116c898564637905c7d23a3f5996b"
server_key_iv: "cb1e14412da8ba384ed12c1aa4e789b9"
server_key_version: "encryptionkey@0"
I20260810 18:48:36.368009 24721 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260810 18:48:36.370597 24737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:36.371811 24721 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 18:48:36.371948 24721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-2/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-2/wal
uuid: "0519acbb10bf4777b6184374d4994e1a"
format_stamp: "Formatted at 2026-08-10 18:48:36 on dist-test-slave-cbsf"
server_key: "cad116c898564637905c7d23a3f5996b"
server_key_iv: "cb1e14412da8ba384ed12c1aa4e789b9"
server_key_version: "encryptionkey@0"
I20260810 18:48:36.372071 24721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:36.376487 24721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:36.377112 24721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:36.377292 24721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:36.377812 24721 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:36.378690 24721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:36.378757 24721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:36.378821 24721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:36.378873 24721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:36.387331 24721 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.131:43485
I20260810 18:48:36.387352 24872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.131:43485 every 8 connection(s)
I20260810 18:48:36.388185 24721 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
I20260810 18:48:36.388978 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 24721
I20260810 18:48:36.389117 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance
I20260810 18:48:36.389382 19106 external_mini_cluster.cc:1428] Setting key e0fb3ce2b27c6c1dba76570989dfb341
I20260810 18:48:36.394433 24394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:36.390014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:57145 (local address 127.18.168.190:43563)
0810 18:48:36.390206 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:36.390211 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:36.391487 (+ 1276us) server_negotiation.cc:407] Connection header received
0810 18:48:36.392848 (+ 1361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:36.392851 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:36.392893 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:36.392973 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:36.393959 (+ 986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:36.393962 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:36.393965 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:48:36.393977 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:36.393998 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:48:36.394007 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:36.394110 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:36.394261 (+ 151us) server_negotiation.cc:299] Negotiation successful
0810 18:48:36.394306 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260810 18:48:36.395294 24875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:36.390489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43563 (local address 127.18.168.131:57145)
0810 18:48:36.391299 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:36.391382 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:36.392591 (+ 1209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:36.393133 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:36.393140 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:36.393232 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:36.393717 (+ 485us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:48:36.393718 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:36.393757 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:48:36.393820 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:36.394097 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:36.394106 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:36.394112 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 18:48:36.394293 (+ 181us) client_negotiation.cc:240] Negotiation successful
0810 18:48:36.394527 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":176,"threads_started":1}
I20260810 18:48:36.396329 24873 heartbeater.cc:347] Connected to a master server at 127.18.168.190:43563
I20260810 18:48:36.396548 24873 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:36.396984 24873 heartbeater.cc:511] Master 127.18.168.190:43563 requested a full tablet report, sending...
I20260810 18:48:36.398032 24351 ts_manager.cc:194] Registered new tserver with Master: 0519acbb10bf4777b6184374d4994e1a (127.18.168.131:43485)
I20260810 18:48:36.403096 19106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:48:36.406394 24394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:36.404983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46656 (local address 127.18.168.190:43563)
0810 18:48:36.405130 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:36.405134 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:36.405327 (+ 193us) server_negotiation.cc:407] Connection header received
0810 18:48:36.405409 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:36.405411 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:36.405440 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:36.405506 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:36.406106 (+ 600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:36.406109 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:36.406110 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:48:36.406120 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:36.406137 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:48:36.406144 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:36.406212 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:36.406277 (+ 65us) server_negotiation.cc:299] Negotiation successful
0810 18:48:36.406306 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 18:48:36.409229 24351 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46656:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:48:36.412155 24351 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:48:36.423830 24884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:36.420534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.131:43485 (local address 127.0.0.1:57870)
0810 18:48:36.421718 (+ 1184us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:36.421731 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:36.421823 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:36.422254 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:36.422261 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:36.422331 (+ 70us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:36.422945 (+ 614us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:48:36.422948 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:36.422970 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:48:36.423042 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:36.423455 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:36.423466 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:36.423472 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 18:48:36.423627 (+ 155us) client_negotiation.cc:240] Negotiation successful
0810 18:48:36.423695 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1115,"thread_start_us":96,"threads_started":1}
I20260810 18:48:36.424055 24885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:36.420811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:36163 (local address 127.0.0.1:40678)
0810 18:48:36.421940 (+ 1129us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:36.421953 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:36.422025 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:36.422996 (+ 971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:36.422999 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:36.423020 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:36.423402 (+ 382us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:48:36.423405 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:36.423413 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:48:36.423440 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:36.423817 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:36.423820 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:36.423823 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 18:48:36.423896 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 18:48:36.423932 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1068,"thread_start_us":83,"threads_started":1}
I20260810 18:48:36.424422 24889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:36.421305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57870 (local address 127.18.168.131:43485)
0810 18:48:36.421733 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:36.421744 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:36.421816 (+ 72us) server_negotiation.cc:407] Connection header received
0810 18:48:36.421906 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:36.421911 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:36.422105 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:36.422253 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:36.423225 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:36.423232 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:36.423245 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:48:36.423258 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:36.423308 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:48:36.423340 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:36.423477 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:36.423644 (+ 167us) server_negotiation.cc:299] Negotiation successful
0810 18:48:36.423715 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":129,"threads_started":1}
I20260810 18:48:36.424458 24890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:36.421322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40678 (local address 127.18.168.130:36163)
0810 18:48:36.422076 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:36.422087 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:36.422108 (+ 21us) server_negotiation.cc:407] Connection header received
0810 18:48:36.422169 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:36.422174 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:36.422331 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:36.422465 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:36.423554 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:36.423560 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:36.423573 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:48:36.423585 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:36.423627 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:48:36.423659 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:36.423768 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:36.424245 (+ 477us) server_negotiation.cc:299] Negotiation successful
0810 18:48:36.424317 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":120,"threads_started":1}
I20260810 18:48:36.424501 24886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:36.420555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40732 (local address 127.18.168.129:44431)
0810 18:48:36.421024 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:36.421037 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:36.421660 (+ 623us) server_negotiation.cc:407] Connection header received
0810 18:48:36.421725 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:36.421733 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:36.421931 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:36.422060 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:36.423586 (+ 1526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:36.423592 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:36.423600 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:48:36.423611 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:36.423660 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:48:36.423697 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:36.423789 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:36.424303 (+ 514us) server_negotiation.cc:299] Negotiation successful
0810 18:48:36.424358 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":135,"threads_started":1}
I20260810 18:48:36.424753 24888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:36.420402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:44431 (local address 127.0.0.1:40732)
0810 18:48:36.421363 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:36.421418 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:36.421527 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:36.422756 (+ 1229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:36.422761 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:36.422777 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:36.423245 (+ 468us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:48:36.423246 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:36.423255 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:48:36.423298 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:36.423824 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:36.423826 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:36.423828 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 18:48:36.423896 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 18:48:36.424063 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":796,"thread_start_us":161,"threads_started":1}
I20260810 18:48:36.427291 24489 tablet_service.cc:1467] Processing CreateTablet for tablet b16741ab5bf24eefa4326af989102a72 (DEFAULT_TABLE table=test-table [id=3286a67d63564bd585484ed7ada54662]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:36.427325 24639 tablet_service.cc:1467] Processing CreateTablet for tablet b16741ab5bf24eefa4326af989102a72 (DEFAULT_TABLE table=test-table [id=3286a67d63564bd585484ed7ada54662]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:36.427325 24794 tablet_service.cc:1467] Processing CreateTablet for tablet b16741ab5bf24eefa4326af989102a72 (DEFAULT_TABLE table=test-table [id=3286a67d63564bd585484ed7ada54662]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:36.428344 24794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b16741ab5bf24eefa4326af989102a72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:36.428345 24639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b16741ab5bf24eefa4326af989102a72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:36.428350 24489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b16741ab5bf24eefa4326af989102a72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:36.434408 24891 tablet_bootstrap.cc:492] T b16741ab5bf24eefa4326af989102a72 P 37a52916ede44230b681def2312027df: Bootstrap starting.
I20260810 18:48:36.436075 24892 tablet_bootstrap.cc:492] T b16741ab5bf24eefa4326af989102a72 P 0519acbb10bf4777b6184374d4994e1a: Bootstrap starting.
I20260810 18:48:36.436851 24891 tablet_bootstrap.cc:654] T b16741ab5bf24eefa4326af989102a72 P 37a52916ede44230b681def2312027df: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:36.437150 24893 tablet_bootstrap.cc:492] T b16741ab5bf24eefa4326af989102a72 P 4ec20fce80b5414185b09bd097d8f57e: Bootstrap starting.
I20260810 18:48:36.437780 24891 log.cc:824] T b16741ab5bf24eefa4326af989102a72 P 37a52916ede44230b681def2312027df: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:36.438089 24892 tablet_bootstrap.cc:654] T b16741ab5bf24eefa4326af989102a72 P 0519acbb10bf4777b6184374d4994e1a: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:36.438864 24892 log.cc:824] T b16741ab5bf24eefa4326af989102a72 P 0519acbb10bf4777b6184374d4994e1a: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:36.439232 24893 tablet_bootstrap.cc:654] T b16741ab5bf24eefa4326af989102a72 P 4ec20fce80b5414185b09bd097d8f57e: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:36.440023 24891 tablet_bootstrap.cc:492] T b16741ab5bf24eefa4326af989102a72 P 37a52916ede44230b681def2312027df: No bootstrap required, opened a new log
I20260810 18:48:36.440032 24893 log.cc:824] T b16741ab5bf24eefa4326af989102a72 P 4ec20fce80b5414185b09bd097d8f57e: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:36.440224 24891 ts_tablet_manager.cc:1397] T b16741ab5bf24eefa4326af989102a72 P 37a52916ede44230b681def2312027df: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260810 18:48:36.440785 24892 tablet_bootstrap.cc:492] T b16741ab5bf24eefa4326af989102a72 P 0519acbb10bf4777b6184374d4994e1a: No bootstrap required, opened a new log
I20260810 18:48:36.440981 24892 ts_tablet_manager.cc:1397] T b16741ab5bf24eefa4326af989102a72 P 0519acbb10bf4777b6184374d4994e1a: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 18:48:36.442395 24893 tablet_bootstrap.cc:492] T b16741ab5bf24eefa4326af989102a72 P 4ec20fce80b5414185b09bd097d8f57e: No bootstrap required, opened a new log
I20260810 18:48:36.442624 24893 ts_tablet_manager.cc:1397] T b16741ab5bf24eefa4326af989102a72 P 4ec20fce80b5414185b09bd097d8f57e: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260810 18:48:36.444433 24892 raft_consensus.cc:348] T b16741ab5bf24eefa4326af989102a72 P 0519acbb10bf4777b6184374d4994e1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37a52916ede44230b681def2312027df" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44431 } } peers { permanent_uuid: "0519acbb10bf4777b6184374d4994e1a" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43485 } } peers { permanent_uuid: "4ec20fce80b5414185b09bd097d8f57e" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36163 } }
I20260810 18:48:36.444674 24892 raft_consensus.cc:374] T b16741ab5bf24eefa4326af989102a72 P 0519acbb10bf4777b6184374d4994e1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:36.444757 24892 raft_consensus.cc:729] T b16741ab5bf24eefa4326af989102a72 P 0519acbb10bf4777b6184374d4994e1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0519acbb10bf4777b6184374d4994e1a, State: Initialized, Role: FOLLOWER
I20260810 18:48:36.444847 24891 raft_consensus.cc:348] T b16741ab5bf24eefa4326af989102a72 P 37a52916ede44230b681def2312027df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37a52916ede44230b681def2312027df" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44431 } } peers { permanent_uuid: "0519acbb10bf4777b6184374d4994e1a" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43485 } } peers { permanent_uuid: "4ec20fce80b5414185b09bd097d8f57e" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36163 } }
I20260810 18:48:36.445091 24891 raft_consensus.cc:374] T b16741ab5bf24eefa4326af989102a72 P 37a52916ede44230b681def2312027df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:36.445158 24891 raft_consensus.cc:729] T b16741ab5bf24eefa4326af989102a72 P 37a52916ede44230b681def2312027df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37a52916ede44230b681def2312027df, State: Initialized, Role: FOLLOWER
I20260810 18:48:36.445236 24892 consensus_queue.cc:260] T b16741ab5bf24eefa4326af989102a72 P 0519acbb10bf4777b6184374d4994e1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37a52916ede44230b681def2312027df" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44431 } } peers { permanent_uuid: "0519acbb10bf4777b6184374d4994e1a" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43485 } } peers { permanent_uuid: "4ec20fce80b5414185b09bd097d8f57e" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36163 } }
I20260810 18:48:36.445612 24891 consensus_queue.cc:260] T b16741ab5bf24eefa4326af989102a72 P 37a52916ede44230b681def2312027df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37a52916ede44230b681def2312027df" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44431 } } peers { permanent_uuid: "0519acbb10bf4777b6184374d4994e1a" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43485 } } peers { permanent_uuid: "4ec20fce80b5414185b09bd097d8f57e" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36163 } }
I20260810 18:48:36.445873 24873 heartbeater.cc:503] Master 127.18.168.190:43563 was elected leader, sending a full tablet report...
I20260810 18:48:36.446214 24892 ts_tablet_manager.cc:1428] T b16741ab5bf24eefa4326af989102a72 P 0519acbb10bf4777b6184374d4994e1a: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20260810 18:48:36.446564 24891 ts_tablet_manager.cc:1428] T b16741ab5bf24eefa4326af989102a72 P 37a52916ede44230b681def2312027df: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 18:48:36.446918 24564 heartbeater.cc:503] Master 127.18.168.190:43563 was elected leader, sending a full tablet report...
I20260810 18:48:36.447005 24893 raft_consensus.cc:348] T b16741ab5bf24eefa4326af989102a72 P 4ec20fce80b5414185b09bd097d8f57e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37a52916ede44230b681def2312027df" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44431 } } peers { permanent_uuid: "0519acbb10bf4777b6184374d4994e1a" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43485 } } peers { permanent_uuid: "4ec20fce80b5414185b09bd097d8f57e" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36163 } }
I20260810 18:48:36.447254 24893 raft_consensus.cc:374] T b16741ab5bf24eefa4326af989102a72 P 4ec20fce80b5414185b09bd097d8f57e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:36.447311 24893 raft_consensus.cc:729] T b16741ab5bf24eefa4326af989102a72 P 4ec20fce80b5414185b09bd097d8f57e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ec20fce80b5414185b09bd097d8f57e, State: Initialized, Role: FOLLOWER
I20260810 18:48:36.447732 24893 consensus_queue.cc:260] T b16741ab5bf24eefa4326af989102a72 P 4ec20fce80b5414185b09bd097d8f57e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37a52916ede44230b681def2312027df" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44431 } } peers { permanent_uuid: "0519acbb10bf4777b6184374d4994e1a" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43485 } } peers { permanent_uuid: "4ec20fce80b5414185b09bd097d8f57e" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36163 } }
I20260810 18:48:36.448490 24717 heartbeater.cc:503] Master 127.18.168.190:43563 was elected leader, sending a full tablet report...
I20260810 18:48:36.449210 24893 ts_tablet_manager.cc:1428] T b16741ab5bf24eefa4326af989102a72 P 4ec20fce80b5414185b09bd097d8f57e: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260810 18:48:36.461896 24718 tablet.cc:2369] T b16741ab5bf24eefa4326af989102a72 P 4ec20fce80b5414185b09bd097d8f57e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:48:36.539152 24565 tablet.cc:2369] T b16741ab5bf24eefa4326af989102a72 P 37a52916ede44230b681def2312027df: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:48:36.639279 24874 tablet.cc:2369] T b16741ab5bf24eefa4326af989102a72 P 0519acbb10bf4777b6184374d4994e1a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:48:36.663846 24900 raft_consensus.cc:482] T b16741ab5bf24eefa4326af989102a72 P 4ec20fce80b5414185b09bd097d8f57e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:48:36.664047 24900 raft_consensus.cc:504] T b16741ab5bf24eefa4326af989102a72 P 4ec20fce80b5414185b09bd097d8f57e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37a52916ede44230b681def2312027df" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44431 } } peers { permanent_uuid: "0519acbb10bf4777b6184374d4994e1a" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43485 } } peers { permanent_uuid: "4ec20fce80b5414185b09bd097d8f57e" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36163 } }
I20260810 18:48:36.665306 24900 leader_election.cc:288] T b16741ab5bf24eefa4326af989102a72 P 4ec20fce80b5414185b09bd097d8f57e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 37a52916ede44230b681def2312027df (127.18.168.129:44431), 0519acbb10bf4777b6184374d4994e1a (127.18.168.131:43485)
I20260810 18:48:36.667333 24719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:36.665553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:44431 (local address 127.18.168.130:47179)
0810 18:48:36.665739 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:36.665755 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:36.665842 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:36.666191 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:36.666195 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:36.666213 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:36.666693 (+ 480us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:48:36.666694 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:36.666704 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:48:36.666728 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:36.667056 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:36.667063 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:36.667067 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 18:48:36.667167 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 18:48:36.667205 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108}
I20260810 18:48:36.667335 24886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:36.665703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:47179 (local address 127.18.168.129:44431)
0810 18:48:36.665957 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:36.665961 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:36.665976 (+ 15us) server_negotiation.cc:407] Connection header received
0810 18:48:36.666022 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:36.666026 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:36.666067 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:36.666144 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:36.666898 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:36.666901 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:36.666906 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:48:36.666922 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:36.666942 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:48:36.666950 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:36.667046 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:36.667171 (+ 125us) server_negotiation.cc:299] Negotiation successful
0810 18:48:36.667211 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148}
I20260810 18:48:36.667490 24889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:36.665675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:35859 (local address 127.18.168.131:43485)
0810 18:48:36.665844 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:36.665849 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:36.665979 (+ 130us) server_negotiation.cc:407] Connection header received
0810 18:48:36.666084 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:36.666086 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:36.666118 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:36.666223 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:36.667103 (+ 880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:36.667106 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:36.667109 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:48:36.667122 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:36.667144 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:48:36.667154 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:36.667249 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:36.667328 (+ 79us) server_negotiation.cc:299] Negotiation successful
0810 18:48:36.667366 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260810 18:48:36.667513 24901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:36.665574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.131:43485 (local address 127.18.168.130:35859)
0810 18:48:36.665877 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:36.665892 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:36.665990 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:36.666271 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:36.666273 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:36.666285 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:36.666936 (+ 651us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:48:36.666938 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:36.666948 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:48:36.666973 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:36.667245 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:36.667249 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:36.667252 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 18:48:36.667330 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 18:48:36.667366 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":131,"threads_started":1}
I20260810 18:48:36.668071 24511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b16741ab5bf24eefa4326af989102a72" candidate_uuid: "4ec20fce80b5414185b09bd097d8f57e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37a52916ede44230b681def2312027df" is_pre_election: true
I20260810 18:48:36.668080 24817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b16741ab5bf24eefa4326af989102a72" candidate_uuid: "4ec20fce80b5414185b09bd097d8f57e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0519acbb10bf4777b6184374d4994e1a" is_pre_election: true
I20260810 18:48:36.668377 24511 raft_consensus.cc:2454] T b16741ab5bf24eefa4326af989102a72 P 37a52916ede44230b681def2312027df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ec20fce80b5414185b09bd097d8f57e in term 0.
I20260810 18:48:36.668377 24817 raft_consensus.cc:2454] T b16741ab5bf24eefa4326af989102a72 P 0519acbb10bf4777b6184374d4994e1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ec20fce80b5414185b09bd097d8f57e in term 0.
I20260810 18:48:36.668905 24584 leader_election.cc:302] T b16741ab5bf24eefa4326af989102a72 P 4ec20fce80b5414185b09bd097d8f57e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0519acbb10bf4777b6184374d4994e1a, 4ec20fce80b5414185b09bd097d8f57e; no voters:
I20260810 18:48:36.669224 24900 raft_consensus.cc:2781] T b16741ab5bf24eefa4326af989102a72 P 4ec20fce80b5414185b09bd097d8f57e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:48:36.669342 24900 raft_consensus.cc:482] T b16741ab5bf24eefa4326af989102a72 P 4ec20fce80b5414185b09bd097d8f57e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:48:36.669402 24900 raft_consensus.cc:3037] T b16741ab5bf24eefa4326af989102a72 P 4ec20fce80b5414185b09bd097d8f57e [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:36.670463 24900 raft_consensus.cc:504] T b16741ab5bf24eefa4326af989102a72 P 4ec20fce80b5414185b09bd097d8f57e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37a52916ede44230b681def2312027df" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44431 } } peers { permanent_uuid: "0519acbb10bf4777b6184374d4994e1a" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43485 } } peers { permanent_uuid: "4ec20fce80b5414185b09bd097d8f57e" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36163 } }
I20260810 18:48:36.670814 24900 leader_election.cc:288] T b16741ab5bf24eefa4326af989102a72 P 4ec20fce80b5414185b09bd097d8f57e [CANDIDATE]: Term 1 election: Requested vote from peers 37a52916ede44230b681def2312027df (127.18.168.129:44431), 0519acbb10bf4777b6184374d4994e1a (127.18.168.131:43485)
I20260810 18:48:36.671187 24817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b16741ab5bf24eefa4326af989102a72" candidate_uuid: "4ec20fce80b5414185b09bd097d8f57e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0519acbb10bf4777b6184374d4994e1a"
I20260810 18:48:36.671206 24511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b16741ab5bf24eefa4326af989102a72" candidate_uuid: "4ec20fce80b5414185b09bd097d8f57e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37a52916ede44230b681def2312027df"
I20260810 18:48:36.671319 24511 raft_consensus.cc:3037] T b16741ab5bf24eefa4326af989102a72 P 37a52916ede44230b681def2312027df [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:36.671315 24817 raft_consensus.cc:3037] T b16741ab5bf24eefa4326af989102a72 P 0519acbb10bf4777b6184374d4994e1a [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:36.672595 24817 raft_consensus.cc:2454] T b16741ab5bf24eefa4326af989102a72 P 0519acbb10bf4777b6184374d4994e1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ec20fce80b5414185b09bd097d8f57e in term 1.
I20260810 18:48:36.672595 24511 raft_consensus.cc:2454] T b16741ab5bf24eefa4326af989102a72 P 37a52916ede44230b681def2312027df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ec20fce80b5414185b09bd097d8f57e in term 1.
I20260810 18:48:36.672920 24585 leader_election.cc:302] T b16741ab5bf24eefa4326af989102a72 P 4ec20fce80b5414185b09bd097d8f57e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37a52916ede44230b681def2312027df, 4ec20fce80b5414185b09bd097d8f57e; no voters:
I20260810 18:48:36.673106 24900 raft_consensus.cc:2781] T b16741ab5bf24eefa4326af989102a72 P 4ec20fce80b5414185b09bd097d8f57e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:36.673342 24900 raft_consensus.cc:686] T b16741ab5bf24eefa4326af989102a72 P 4ec20fce80b5414185b09bd097d8f57e [term 1 LEADER]: Becoming Leader. State: Replica: 4ec20fce80b5414185b09bd097d8f57e, State: Running, Role: LEADER
I20260810 18:48:36.673682 24900 consensus_queue.cc:237] T b16741ab5bf24eefa4326af989102a72 P 4ec20fce80b5414185b09bd097d8f57e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37a52916ede44230b681def2312027df" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44431 } } peers { permanent_uuid: "0519acbb10bf4777b6184374d4994e1a" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43485 } } peers { permanent_uuid: "4ec20fce80b5414185b09bd097d8f57e" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36163 } }
I20260810 18:48:36.676872 24351 catalog_manager.cc:5495] T b16741ab5bf24eefa4326af989102a72 P 4ec20fce80b5414185b09bd097d8f57e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ec20fce80b5414185b09bd097d8f57e (127.18.168.130). New cstate: current_term: 1 leader_uuid: "4ec20fce80b5414185b09bd097d8f57e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37a52916ede44230b681def2312027df" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0519acbb10bf4777b6184374d4994e1a" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ec20fce80b5414185b09bd097d8f57e" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36163 } health_report { overall_health: HEALTHY } } }
I20260810 18:48:36.715272 24890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:36.713667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40680 (local address 127.18.168.130:36163)
0810 18:48:36.713799 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:36.713805 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:36.713818 (+ 13us) server_negotiation.cc:407] Connection header received
0810 18:48:36.714001 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:36.714003 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:36.714061 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:36.714149 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:36.714886 (+ 737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:36.714890 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:36.714893 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:48:36.714904 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:36.714932 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:48:36.714940 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:36.715034 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:36.715111 (+ 77us) server_negotiation.cc:299] Negotiation successful
0810 18:48:36.715149 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 18:48:36.720816 24817 raft_consensus.cc:1261] T b16741ab5bf24eefa4326af989102a72 P 0519acbb10bf4777b6184374d4994e1a [term 1 FOLLOWER]: Refusing update from remote peer 4ec20fce80b5414185b09bd097d8f57e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:36.720872 24511 raft_consensus.cc:1261] T b16741ab5bf24eefa4326af989102a72 P 37a52916ede44230b681def2312027df [term 1 FOLLOWER]: Refusing update from remote peer 4ec20fce80b5414185b09bd097d8f57e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:36.721551 24900 consensus_queue.cc:1035] T b16741ab5bf24eefa4326af989102a72 P 4ec20fce80b5414185b09bd097d8f57e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0519acbb10bf4777b6184374d4994e1a" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:36.721735 24902 consensus_queue.cc:1035] T b16741ab5bf24eefa4326af989102a72 P 4ec20fce80b5414185b09bd097d8f57e [LEADER]: Connected to new peer: Peer: permanent_uuid: "37a52916ede44230b681def2312027df" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:36.736330 24351 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46656:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:36.736503 24351 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46656:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:36.738950 24351 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 94788d10774c4ccc8e74a6ecc8778206: Sending DeleteTablet for 3 replicas of tablet b16741ab5bf24eefa4326af989102a72
I20260810 18:48:36.739706 24489 tablet_service.cc:1514] Processing DeleteTablet for tablet b16741ab5bf24eefa4326af989102a72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:36 UTC) from {username='slave'} at 127.0.0.1:40732
I20260810 18:48:36.739709 24794 tablet_service.cc:1514] Processing DeleteTablet for tablet b16741ab5bf24eefa4326af989102a72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:36 UTC) from {username='slave'} at 127.0.0.1:57870
I20260810 18:48:36.739859 24639 tablet_service.cc:1514] Processing DeleteTablet for tablet b16741ab5bf24eefa4326af989102a72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:36 UTC) from {username='slave'} at 127.0.0.1:40678
I20260810 18:48:36.740167 24916 tablet_replica.cc:330] T b16741ab5bf24eefa4326af989102a72 P 37a52916ede44230b681def2312027df: stopping tablet replica
I20260810 18:48:36.740415 24917 tablet_replica.cc:330] T b16741ab5bf24eefa4326af989102a72 P 4ec20fce80b5414185b09bd097d8f57e: stopping tablet replica
I20260810 18:48:36.740476 24916 raft_consensus.cc:2229] T b16741ab5bf24eefa4326af989102a72 P 37a52916ede44230b681def2312027df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:36.740826 24917 raft_consensus.cc:2229] T b16741ab5bf24eefa4326af989102a72 P 4ec20fce80b5414185b09bd097d8f57e [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:48:36.741014 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 24416
I20260810 18:48:36.757965 24917 raft_consensus.cc:2258] T b16741ab5bf24eefa4326af989102a72 P 4ec20fce80b5414185b09bd097d8f57e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:36.758436 24915 tablet_replica.cc:330] T b16741ab5bf24eefa4326af989102a72 P 0519acbb10bf4777b6184374d4994e1a: stopping tablet replica
I20260810 18:48:36.758749 24915 raft_consensus.cc:2229] T b16741ab5bf24eefa4326af989102a72 P 0519acbb10bf4777b6184374d4994e1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:36.759068 24915 raft_consensus.cc:2258] T b16741ab5bf24eefa4326af989102a72 P 0519acbb10bf4777b6184374d4994e1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:36.760072 24917 ts_tablet_manager.cc:1905] T b16741ab5bf24eefa4326af989102a72 P 4ec20fce80b5414185b09bd097d8f57e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:36.760494 24915 ts_tablet_manager.cc:1905] T b16741ab5bf24eefa4326af989102a72 P 0519acbb10bf4777b6184374d4994e1a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:36.763170 24915 ts_tablet_manager.cc:1918] T b16741ab5bf24eefa4326af989102a72 P 0519acbb10bf4777b6184374d4994e1a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:48:36.763165 24917 ts_tablet_manager.cc:1918] T b16741ab5bf24eefa4326af989102a72 P 4ec20fce80b5414185b09bd097d8f57e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:48:36.763286 24915 log.cc:1196] T b16741ab5bf24eefa4326af989102a72 P 0519acbb10bf4777b6184374d4994e1a: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-2/wal/wals/b16741ab5bf24eefa4326af989102a72
I20260810 18:48:36.763409 24917 log.cc:1196] T b16741ab5bf24eefa4326af989102a72 P 4ec20fce80b5414185b09bd097d8f57e: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387687857264-19106-0/minicluster-data/ts-1/wal/wals/b16741ab5bf24eefa4326af989102a72
I20260810 18:48:36.763682 24915 ts_tablet_manager.cc:1939] T b16741ab5bf24eefa4326af989102a72 P 0519acbb10bf4777b6184374d4994e1a: Deleting consensus metadata
I20260810 18:48:36.763927 24917 ts_tablet_manager.cc:1939] T b16741ab5bf24eefa4326af989102a72 P 4ec20fce80b5414185b09bd097d8f57e: Deleting consensus metadata
W20260810 18:48:36.764966 24331 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.168.129:44431 (error 108)
I20260810 18:48:36.765069 24330 catalog_manager.cc:4841] TS 0519acbb10bf4777b6184374d4994e1a (127.18.168.131:43485): tablet b16741ab5bf24eefa4326af989102a72 (table test-table [id=3286a67d63564bd585484ed7ada54662]) successfully deleted
I20260810 18:48:36.765079 24332 catalog_manager.cc:4841] TS 4ec20fce80b5414185b09bd097d8f57e (127.18.168.130:36163): tablet b16741ab5bf24eefa4326af989102a72 (table test-table [id=3286a67d63564bd585484ed7ada54662]) successfully deleted
I20260810 18:48:36.765800 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 24567
I20260810 18:48:36.766165 24888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:36.765837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:44431 (local address 127.0.0.1:40748)
0810 18:48:36.766009 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:36.766077 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.129:44431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
W20260810 18:48:36.766320 24331 catalog_manager.cc:4584] TS 37a52916ede44230b681def2312027df (127.18.168.129:44431): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b16741ab5bf24eefa4326af989102a72: Network error: Client connection negotiation failed: client connection to 127.18.168.129:44431: connect: Connection refused (error 111)
I20260810 18:48:36.774035 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 24721
I20260810 18:48:36.781749 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 24314
2026-08-10T18:48:36Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1154 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260810 18:48:36.805080 19106 mini_oidc.cc:152] Starting JWKS server
I20260810 18:48:36.806779 19106 webserver.cc:460] Webserver started at https://127.0.0.1:39979/ using document root <none> and password file <none>
I20260810 18:48:36.806944 19106 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 18:48:36.807226 19106 webserver.cc:460] Webserver started at http://127.0.0.1:46861/ using document root <none> and password file <none>
2026-08-10T18:48:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:48:36Z Disabled control of system clock
I20260810 18:48:36.817210 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:35069
--webserver_interface=127.18.168.190
--webserver_port=0
--builtin_ntp_servers=127.18.168.148:40019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:35069
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39979/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387687857264-19106-0/testchainca.cert with env {}
W20260810 18:48:36.927649 24924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:36.927881 24924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:36.927917 24924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:36.931825 24924 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:48:36.931905 24924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:36.931931 24924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:36.931962 24924 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:48:36.931988 24924 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 18:48:36.932006 24924 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 18:48:36.932034 24924 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39979/jwks.json
I20260810 18:48:36.937136 24924 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:40019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:35069
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:35069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39979/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387687857264-19106-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:36.938071 24924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:36.938993 24924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:36.946928 24934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:36.946934 24931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:36.946935 24932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:36.947278 24924 server_base.cc:1034] running on GCE node
I20260810 18:48:36.947793 24924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:36.948865 24924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:36.950050 24924 hybrid_clock.cc:648] HybridClock initialized: now 1786387716950033 us; error 28 us; skew 500 ppm
I20260810 18:48:36.951910 24924 webserver.cc:460] Webserver started at http://127.18.168.190:36789/ using document root <none> and password file <none>
I20260810 18:48:36.952543 24924 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:36.952716 24924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:36.952975 24924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:36.955090 24924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "17924861db44461287589655f47dc742"
format_stamp: "Formatted at 2026-08-10 18:48:36 on dist-test-slave-cbsf"
server_key: "e9ef6a6f87dbe53aa90ae37e2959533d"
server_key_iv: "8039c0d108f212ca5dc69d7f8d43cac5"
server_key_version: "encryptionkey@0"
I20260810 18:48:36.955672 24924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "17924861db44461287589655f47dc742"
format_stamp: "Formatted at 2026-08-10 18:48:36 on dist-test-slave-cbsf"
server_key: "e9ef6a6f87dbe53aa90ae37e2959533d"
server_key_iv: "8039c0d108f212ca5dc69d7f8d43cac5"
server_key_version: "encryptionkey@0"
I20260810 18:48:36.959257 24924 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260810 18:48:36.962075 24940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:36.963359 24924 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:48:36.963456 24924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "17924861db44461287589655f47dc742"
format_stamp: "Formatted at 2026-08-10 18:48:36 on dist-test-slave-cbsf"
server_key: "e9ef6a6f87dbe53aa90ae37e2959533d"
server_key_iv: "8039c0d108f212ca5dc69d7f8d43cac5"
server_key_version: "encryptionkey@0"
I20260810 18:48:36.963533 24924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:36.998633 24924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:36.999310 24924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:36.999493 24924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:37.006959 24924 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:35069
I20260810 18:48:37.006986 24997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:35069 every 8 connection(s)
I20260810 18:48:37.007920 24924 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:48:37.011293 25000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:37.013322 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 24924
I20260810 18:48:37.013447 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:48:37.013768 19106 external_mini_cluster.cc:1428] Setting key c3c54045adf1cf108320c95403737917
I20260810 18:48:37.017781 25000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17924861db44461287589655f47dc742: Bootstrap starting.
I20260810 18:48:37.020341 25000 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17924861db44461287589655f47dc742: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:37.021196 25000 log.cc:824] T 00000000000000000000000000000000 P 17924861db44461287589655f47dc742: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:37.023411 25000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17924861db44461287589655f47dc742: No bootstrap required, opened a new log
I20260810 18:48:37.023672 25004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:37.015201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42556 (local address 127.18.168.190:35069)
0810 18:48:37.015830 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:37.015846 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:37.015881 (+ 35us) server_negotiation.cc:407] Connection header received
0810 18:48:37.016655 (+ 774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:37.016673 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:37.017263 (+ 590us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:37.017648 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:37.018114 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:37.019095 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:37.020071 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:37.020626 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:37.020841 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:37.021674 (+ 833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:37.021680 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:37.021697 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:48:37.022183 (+ 486us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:37.022255 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:48:37.022282 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:37.022383 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:37.022551 (+ 168us) server_negotiation.cc:299] Negotiation successful
0810 18:48:37.022801 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":134,"threads_started":1}
I20260810 18:48:37.026491 19106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:48:37.027639 25000 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17924861db44461287589655f47dc742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17924861db44461287589655f47dc742" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 35069 } }
I20260810 18:48:37.027839 25000 raft_consensus.cc:374] T 00000000000000000000000000000000 P 17924861db44461287589655f47dc742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:37.027892 25000 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17924861db44461287589655f47dc742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17924861db44461287589655f47dc742, State: Initialized, Role: FOLLOWER
I20260810 18:48:37.028321 25000 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17924861db44461287589655f47dc742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17924861db44461287589655f47dc742" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 35069 } }
I20260810 18:48:37.028469 25000 raft_consensus.cc:388] T 00000000000000000000000000000000 P 17924861db44461287589655f47dc742 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:48:37.028558 25000 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17924861db44461287589655f47dc742 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:48:37.028671 25000 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17924861db44461287589655f47dc742 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:37.029573 25000 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17924861db44461287589655f47dc742 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17924861db44461287589655f47dc742" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 35069 } }
I20260810 18:48:37.029897 25000 leader_election.cc:302] T 00000000000000000000000000000000 P 17924861db44461287589655f47dc742 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17924861db44461287589655f47dc742; no voters:
I20260810 18:48:37.030194 25000 leader_election.cc:288] T 00000000000000000000000000000000 P 17924861db44461287589655f47dc742 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:48:37.030323 25006 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17924861db44461287589655f47dc742 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:37.030571 25006 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17924861db44461287589655f47dc742 [term 1 LEADER]: Becoming Leader. State: Replica: 17924861db44461287589655f47dc742, State: Running, Role: LEADER
I20260810 18:48:37.030876 25006 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17924861db44461287589655f47dc742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17924861db44461287589655f47dc742" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 35069 } }
I20260810 18:48:37.031412 25000 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17924861db44461287589655f47dc742 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:48:37.032473 25009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17924861db44461287589655f47dc742 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "17924861db44461287589655f47dc742" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17924861db44461287589655f47dc742" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 35069 } } }
I20260810 18:48:37.032612 25009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17924861db44461287589655f47dc742 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:37.032920 25010 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17924861db44461287589655f47dc742 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17924861db44461287589655f47dc742. Latest consensus state: current_term: 1 leader_uuid: "17924861db44461287589655f47dc742" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17924861db44461287589655f47dc742" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 35069 } } }
I20260810 18:48:37.033034 25010 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17924861db44461287589655f47dc742 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:37.033181 25019 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:48:37.036392 25019 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:48:37.042243 25019 catalog_manager.cc:1347] Generated new cluster ID: 01894ab7f238473082c4e02bcf5dcc9f
I20260810 18:48:37.042311 25019 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:48:37.060770 25019 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:48:37.062055 25019 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:48:37.087877 25019 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 17924861db44461287589655f47dc742: Generated new TSK 0
I20260810 18:48:37.088713 25019 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 18:48:37.121846 25038 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260810 18:48:37.122062 25038 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 18:48:37.122710 25004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:37.100910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42558 (local address 127.18.168.190:35069)
0810 18:48:37.101078 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:37.101083 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:37.101285 (+ 202us) server_negotiation.cc:407] Connection header received
0810 18:48:37.101547 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:37.101551 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:37.101610 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:37.101714 (+ 104us) server_negotiation.cc:226] Negotiated authn=JWT
0810 18:48:37.101985 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:37.102522 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:37.103414 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:37.103651 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:37.103758 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:37.104117 (+ 359us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 18:48:37.122218 (+ 18101us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39979/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 18:48:37.122567 (+ 349us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42558: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39979/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":85}
W20260810 18:48:37.122807 25004 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42558: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39979/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260810 18:48:37.123978 25036 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:37.100847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:35069 (local address 127.0.0.1:42558)
0810 18:48:37.101200 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:37.101212 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:37.101357 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:37.101732 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:37.101736 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:37.101744 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0810 18:48:37.101874 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:37.101880 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:37.102664 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:37.102668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:37.103284 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:37.103291 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:37.103766 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:37.103770 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:37.103862 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:37.103913 (+ 51us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 18:48:37.122470 (+ 18557us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39979/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 18:48:37.122728 (+ 258us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.168.190:35069: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39979/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":120,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.18.168.190:35069: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39979/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260810 18:48:37.124778 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 24924
2026-08-10T18:48:37Z chronyd exiting
I20260810 18:48:37.165767 19106 test_util.cc:175] -----------------------------------------------
I20260810 18:48:37.165866 19106 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387687857264-19106-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (362 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260810 18:48:37.166846 19106 mini_oidc.cc:152] Starting JWKS server
I20260810 18:48:37.167716 19106 webserver.cc:460] Webserver started at https://127.0.0.1:38265/ using document root <none> and password file <none>
I20260810 18:48:37.167838 19106 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 18:48:37.168135 19106 webserver.cc:460] Webserver started at http://127.0.0.1:45745/ using document root <none> and password file <none>
2026-08-10T18:48:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:48:37Z Disabled control of system clock
I20260810 18:48:37.178351 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:42627
--webserver_interface=127.18.168.190
--webserver_port=0
--builtin_ntp_servers=127.18.168.148:43553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:42627
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38265/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387687857264-19106-0/testchainca.cert with env {}
W20260810 18:48:37.291836 25044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:37.292094 25044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:37.292135 25044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:37.295994 25044 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:48:37.296046 25044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:37.296063 25044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:37.296082 25044 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:48:37.296097 25044 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 18:48:37.296121 25044 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 18:48:37.296136 25044 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38265/jwks.json
I20260810 18:48:37.301286 25044 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:43553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:42627
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:42627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38265/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387687857264-19106-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:37.302192 25044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:37.303100 25044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:37.312012 25053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:37.312002 25049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:37.312038 25050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:37.312168 25044 server_base.cc:1034] running on GCE node
I20260810 18:48:37.312924 25044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:37.314213 25044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:37.315514 25044 hybrid_clock.cc:648] HybridClock initialized: now 1786387717315485 us; error 40 us; skew 500 ppm
I20260810 18:48:37.317317 25044 webserver.cc:460] Webserver started at http://127.18.168.190:33253/ using document root <none> and password file <none>
I20260810 18:48:37.317981 25044 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:37.318073 25044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:37.318291 25044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:37.320212 25044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "e6d953378c274da395f57fcb7d5f2247"
format_stamp: "Formatted at 2026-08-10 18:48:37 on dist-test-slave-cbsf"
server_key: "7b3178cdd04b3e8157aa5076ab0a635d"
server_key_iv: "26874fd6511ee8bc754ab65a09858f83"
server_key_version: "encryptionkey@0"
I20260810 18:48:37.320799 25044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "e6d953378c274da395f57fcb7d5f2247"
format_stamp: "Formatted at 2026-08-10 18:48:37 on dist-test-slave-cbsf"
server_key: "7b3178cdd04b3e8157aa5076ab0a635d"
server_key_iv: "26874fd6511ee8bc754ab65a09858f83"
server_key_version: "encryptionkey@0"
I20260810 18:48:37.324251 25044 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:48:37.326526 25060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:37.327704 25044 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 18:48:37.327837 25044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "e6d953378c274da395f57fcb7d5f2247"
format_stamp: "Formatted at 2026-08-10 18:48:37 on dist-test-slave-cbsf"
server_key: "7b3178cdd04b3e8157aa5076ab0a635d"
server_key_iv: "26874fd6511ee8bc754ab65a09858f83"
server_key_version: "encryptionkey@0"
I20260810 18:48:37.327942 25044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:37.347036 25044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:37.347667 25044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:37.347854 25044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:37.354823 25044 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:42627
I20260810 18:48:37.354836 25121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:42627 every 8 connection(s)
I20260810 18:48:37.355867 25044 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:48:37.358908 25122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:37.364290 25122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6d953378c274da395f57fcb7d5f2247: Bootstrap starting.
I20260810 18:48:37.364534 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 25044
I20260810 18:48:37.364671 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:48:37.364998 19106 external_mini_cluster.cc:1428] Setting key 511b52e7fa6114ab7d807a5c81204977
I20260810 18:48:37.367048 25122 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6d953378c274da395f57fcb7d5f2247: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:37.367797 25122 log.cc:824] T 00000000000000000000000000000000 P e6d953378c274da395f57fcb7d5f2247: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:37.369897 25122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6d953378c274da395f57fcb7d5f2247: No bootstrap required, opened a new log
I20260810 18:48:37.373598 25122 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6d953378c274da395f57fcb7d5f2247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6d953378c274da395f57fcb7d5f2247" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 42627 } }
I20260810 18:48:37.373850 25122 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6d953378c274da395f57fcb7d5f2247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:37.373937 25122 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6d953378c274da395f57fcb7d5f2247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6d953378c274da395f57fcb7d5f2247, State: Initialized, Role: FOLLOWER
I20260810 18:48:37.374217 25125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:37.366351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34096 (local address 127.18.168.190:42627)
0810 18:48:37.367003 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:37.367016 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:37.367051 (+ 35us) server_negotiation.cc:407] Connection header received
0810 18:48:37.367740 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:37.367756 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:37.368199 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:37.368587 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:37.368998 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:37.369921 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:37.370759 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:37.371233 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:37.371443 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:37.372317 (+ 874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:37.372322 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:37.372339 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:48:37.372778 (+ 439us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:37.372832 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:48:37.372853 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:37.372961 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:37.373124 (+ 163us) server_negotiation.cc:299] Negotiation successful
0810 18:48:37.373363 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":152,"threads_started":1}
I20260810 18:48:37.374431 25122 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6d953378c274da395f57fcb7d5f2247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6d953378c274da395f57fcb7d5f2247" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 42627 } }
I20260810 18:48:37.374595 25122 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6d953378c274da395f57fcb7d5f2247 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:48:37.374665 25122 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6d953378c274da395f57fcb7d5f2247 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:48:37.374759 25122 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6d953378c274da395f57fcb7d5f2247 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:37.375675 25122 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6d953378c274da395f57fcb7d5f2247 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6d953378c274da395f57fcb7d5f2247" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 42627 } }
I20260810 18:48:37.376055 25122 leader_election.cc:302] T 00000000000000000000000000000000 P e6d953378c274da395f57fcb7d5f2247 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6d953378c274da395f57fcb7d5f2247; no voters:
I20260810 18:48:37.376451 25122 leader_election.cc:288] T 00000000000000000000000000000000 P e6d953378c274da395f57fcb7d5f2247 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:48:37.376530 25129 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6d953378c274da395f57fcb7d5f2247 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:37.376775 25129 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6d953378c274da395f57fcb7d5f2247 [term 1 LEADER]: Becoming Leader. State: Replica: e6d953378c274da395f57fcb7d5f2247, State: Running, Role: LEADER
I20260810 18:48:37.377143 19106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:48:37.377121 25129 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6d953378c274da395f57fcb7d5f2247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6d953378c274da395f57fcb7d5f2247" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 42627 } }
I20260810 18:48:37.377749 25122 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6d953378c274da395f57fcb7d5f2247 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:48:37.379015 25130 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6d953378c274da395f57fcb7d5f2247 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e6d953378c274da395f57fcb7d5f2247" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6d953378c274da395f57fcb7d5f2247" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 42627 } } }
I20260810 18:48:37.379145 25130 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6d953378c274da395f57fcb7d5f2247 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:37.379535 25131 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6d953378c274da395f57fcb7d5f2247 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6d953378c274da395f57fcb7d5f2247. Latest consensus state: current_term: 1 leader_uuid: "e6d953378c274da395f57fcb7d5f2247" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6d953378c274da395f57fcb7d5f2247" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 42627 } } }
I20260810 18:48:37.379638 25131 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6d953378c274da395f57fcb7d5f2247 [sys.catalog]: This master's current role is: LEADER
W20260810 18:48:37.382412 25149 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e6d953378c274da395f57fcb7d5f2247: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:48:37.382478 25149 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:48:37.382560 25141 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:48:37.383289 25141 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:48:37.388782 25141 catalog_manager.cc:1347] Generated new cluster ID: 4b949b542646427b9457643552162614
I20260810 18:48:37.388841 25141 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:48:37.395164 25141 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:48:37.396133 25141 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:48:37.402004 25141 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6d953378c274da395f57fcb7d5f2247: Generated new TSK 0
I20260810 18:48:37.402669 25141 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 18:48:37.429605 25161 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260810 18:48:37.429693 25161 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 18:48:37.430315 25125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:37.414741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34100 (local address 127.18.168.190:42627)
0810 18:48:37.414869 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:37.414874 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:37.415020 (+ 146us) server_negotiation.cc:407] Connection header received
0810 18:48:37.415143 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:37.415146 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:37.415205 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:37.415316 (+ 111us) server_negotiation.cc:226] Negotiated authn=JWT
0810 18:48:37.415558 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:37.416406 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:37.417348 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:37.417701 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:37.417798 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:37.418149 (+ 351us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 18:48:37.429944 (+ 11795us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38265/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 18:48:37.430185 (+ 241us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34100: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38265/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":51}
W20260810 18:48:37.430377 25125 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34100: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38265/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260810 18:48:37.430413 25160 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:37.414624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:42627 (local address 127.0.0.1:34100)
0810 18:48:37.414936 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:37.414948 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:37.415044 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:37.415320 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:37.415322 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:37.415329 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0810 18:48:37.415424 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:37.415429 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:37.416534 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:37.416537 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:37.417231 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:37.417240 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:37.417884 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:37.417887 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:37.417953 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:37.417976 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 18:48:37.430129 (+ 12153us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38265/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 18:48:37.430254 (+ 125us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.168.190:42627: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38265/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":110,"threads_started":1}
I20260810 18:48:37.430987 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 25044
2026-08-10T18:48:37Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (306 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260810 18:48:37.473521 19106 mini_oidc.cc:152] Starting JWKS server
I20260810 18:48:37.474375 19106 webserver.cc:460] Webserver started at https://127.0.0.1:41439/ using document root <none> and password file <none>
I20260810 18:48:37.474488 19106 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 18:48:37.474762 19106 webserver.cc:460] Webserver started at http://127.0.0.1:43361/ using document root <none> and password file <none>
2026-08-10T18:48:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:48:37Z Disabled control of system clock
I20260810 18:48:37.484853 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:37857
--webserver_interface=127.18.168.190
--webserver_port=0
--builtin_ntp_servers=127.18.168.148:34459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:37857
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41439/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 18:48:37.592100 25173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:37.592392 25173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:37.592465 25173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:37.596334 25173 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:48:37.596426 25173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:37.596454 25173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:37.596508 25173 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:48:37.596542 25173 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 18:48:37.596590 25173 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 18:48:37.596619 25173 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41439/jwks.json
I20260810 18:48:37.601263 25173 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:34459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:37857
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:37857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41439/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:37.602279 25173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:37.603734 25173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:37.612156 25179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:37.612196 25178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:37.612166 25182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:37.612216 25173 server_base.cc:1034] running on GCE node
I20260810 18:48:37.612968 25173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:37.613991 25173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:37.615182 25173 hybrid_clock.cc:648] HybridClock initialized: now 1786387717615158 us; error 40 us; skew 500 ppm
I20260810 18:48:37.617003 25173 webserver.cc:460] Webserver started at http://127.18.168.190:41207/ using document root <none> and password file <none>
I20260810 18:48:37.617594 25173 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:37.617689 25173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:37.617911 25173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:37.619923 25173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "273d9a3763e941ae814cd713043d9f11"
format_stamp: "Formatted at 2026-08-10 18:48:37 on dist-test-slave-cbsf"
server_key: "29193d26585e0ffacb886cbeefdc5b12"
server_key_iv: "d5c869036e9b33199247a775d7e2f8c1"
server_key_version: "encryptionkey@0"
I20260810 18:48:37.620515 25173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "273d9a3763e941ae814cd713043d9f11"
format_stamp: "Formatted at 2026-08-10 18:48:37 on dist-test-slave-cbsf"
server_key: "29193d26585e0ffacb886cbeefdc5b12"
server_key_iv: "d5c869036e9b33199247a775d7e2f8c1"
server_key_version: "encryptionkey@0"
I20260810 18:48:37.624048 25173 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:48:37.626370 25188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:37.627478 25173 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:48:37.627620 25173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "273d9a3763e941ae814cd713043d9f11"
format_stamp: "Formatted at 2026-08-10 18:48:37 on dist-test-slave-cbsf"
server_key: "29193d26585e0ffacb886cbeefdc5b12"
server_key_iv: "d5c869036e9b33199247a775d7e2f8c1"
server_key_version: "encryptionkey@0"
I20260810 18:48:37.627728 25173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:37.636199 25173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:37.636850 25173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:37.637044 25173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:37.643733 25173 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:37857
I20260810 18:48:37.643739 25245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:37857 every 8 connection(s)
I20260810 18:48:37.644579 25173 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:48:37.647287 25247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:37.650492 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 25173
I20260810 18:48:37.650645 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:48:37.650951 19106 external_mini_cluster.cc:1428] Setting key 0333170c727425d0e1a24694c5f67138
I20260810 18:48:37.653326 25247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 273d9a3763e941ae814cd713043d9f11: Bootstrap starting.
I20260810 18:48:37.655429 25247 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 273d9a3763e941ae814cd713043d9f11: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:37.656159 25247 log.cc:824] T 00000000000000000000000000000000 P 273d9a3763e941ae814cd713043d9f11: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:37.658042 25247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 273d9a3763e941ae814cd713043d9f11: No bootstrap required, opened a new log
I20260810 18:48:37.660513 25250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:37.652335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47668 (local address 127.18.168.190:37857)
0810 18:48:37.653030 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:37.653046 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:37.653079 (+ 33us) server_negotiation.cc:407] Connection header received
0810 18:48:37.653751 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:37.653770 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:37.654141 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:37.654457 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:37.654914 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:37.655930 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:37.657108 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:37.657583 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:37.657763 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:37.658649 (+ 886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:37.658662 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:37.658674 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:48:37.659038 (+ 364us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:37.659094 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:48:37.659114 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:37.659220 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:37.659353 (+ 133us) server_negotiation.cc:299] Negotiation successful
0810 18:48:37.659605 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":138,"threads_started":1}
I20260810 18:48:37.661191 25247 raft_consensus.cc:348] T 00000000000000000000000000000000 P 273d9a3763e941ae814cd713043d9f11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "273d9a3763e941ae814cd713043d9f11" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 37857 } }
I20260810 18:48:37.661444 25247 raft_consensus.cc:374] T 00000000000000000000000000000000 P 273d9a3763e941ae814cd713043d9f11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:37.661504 25247 raft_consensus.cc:729] T 00000000000000000000000000000000 P 273d9a3763e941ae814cd713043d9f11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 273d9a3763e941ae814cd713043d9f11, State: Initialized, Role: FOLLOWER
I20260810 18:48:37.661963 25247 consensus_queue.cc:260] T 00000000000000000000000000000000 P 273d9a3763e941ae814cd713043d9f11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "273d9a3763e941ae814cd713043d9f11" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 37857 } }
I20260810 18:48:37.662119 25247 raft_consensus.cc:388] T 00000000000000000000000000000000 P 273d9a3763e941ae814cd713043d9f11 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:48:37.662185 25247 raft_consensus.cc:482] T 00000000000000000000000000000000 P 273d9a3763e941ae814cd713043d9f11 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:48:37.662279 25247 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 273d9a3763e941ae814cd713043d9f11 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:37.663290 25247 raft_consensus.cc:504] T 00000000000000000000000000000000 P 273d9a3763e941ae814cd713043d9f11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "273d9a3763e941ae814cd713043d9f11" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 37857 } }
I20260810 18:48:37.663681 19106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:48:37.663695 25247 leader_election.cc:302] T 00000000000000000000000000000000 P 273d9a3763e941ae814cd713043d9f11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 273d9a3763e941ae814cd713043d9f11; no voters:
I20260810 18:48:37.663993 25247 leader_election.cc:288] T 00000000000000000000000000000000 P 273d9a3763e941ae814cd713043d9f11 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:48:37.664109 25252 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 273d9a3763e941ae814cd713043d9f11 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:37.664345 25252 raft_consensus.cc:686] T 00000000000000000000000000000000 P 273d9a3763e941ae814cd713043d9f11 [term 1 LEADER]: Becoming Leader. State: Replica: 273d9a3763e941ae814cd713043d9f11, State: Running, Role: LEADER
I20260810 18:48:37.664673 25252 consensus_queue.cc:237] T 00000000000000000000000000000000 P 273d9a3763e941ae814cd713043d9f11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "273d9a3763e941ae814cd713043d9f11" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 37857 } }
I20260810 18:48:37.665306 25247 sys_catalog.cc:564] T 00000000000000000000000000000000 P 273d9a3763e941ae814cd713043d9f11 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:48:37.666594 25254 sys_catalog.cc:455] T 00000000000000000000000000000000 P 273d9a3763e941ae814cd713043d9f11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 273d9a3763e941ae814cd713043d9f11. Latest consensus state: current_term: 1 leader_uuid: "273d9a3763e941ae814cd713043d9f11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "273d9a3763e941ae814cd713043d9f11" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 37857 } } }
I20260810 18:48:37.666728 25254 sys_catalog.cc:458] T 00000000000000000000000000000000 P 273d9a3763e941ae814cd713043d9f11 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:37.666764 25253 sys_catalog.cc:455] T 00000000000000000000000000000000 P 273d9a3763e941ae814cd713043d9f11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "273d9a3763e941ae814cd713043d9f11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "273d9a3763e941ae814cd713043d9f11" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 37857 } } }
I20260810 18:48:37.666829 25253 sys_catalog.cc:458] T 00000000000000000000000000000000 P 273d9a3763e941ae814cd713043d9f11 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:37.667140 25262 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:48:37.669741 25262 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:48:37.674649 25262 catalog_manager.cc:1347] Generated new cluster ID: f29df83d011948099913fe905f127d4b
I20260810 18:48:37.674712 25262 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:48:37.693898 25262 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:48:37.695099 25262 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:48:37.706336 25262 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 273d9a3763e941ae814cd713043d9f11: Generated new TSK 0
I20260810 18:48:37.707010 25262 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:48:37.722295 25250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:37.716968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47676 (local address 127.18.168.190:37857)
0810 18:48:37.717106 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:37.717111 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:37.717355 (+ 244us) server_negotiation.cc:407] Connection header received
0810 18:48:37.717527 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:37.717529 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:37.717577 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:37.717659 (+ 82us) server_negotiation.cc:226] Negotiated authn=JWT
0810 18:48:37.717912 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:37.718576 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:37.719876 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:37.720086 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:37.720166 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:37.720479 (+ 313us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 18:48:37.721923 (+ 1444us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41439/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 18:48:37.722156 (+ 233us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47676: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41439/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":53}
W20260810 18:48:37.722358 25250 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47676: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41439/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260810 18:48:37.723980 25278 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:37.716917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:37857 (local address 127.0.0.1:47676)
0810 18:48:37.717260 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:37.717276 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:37.717393 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:37.717685 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:37.717688 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:37.717696 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0810 18:48:37.717807 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:37.717814 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:37.718695 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:37.718699 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:37.719761 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:37.719771 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:37.720200 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:37.720204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:37.720295 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:37.720323 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 18:48:37.722095 (+ 1772us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41439/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 18:48:37.723808 (+ 1713us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.168.190:37857: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41439/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":225,"mutex_wait_us":1522,"thread_start_us":117,"threads_started":1}
I20260810 18:48:37.724572 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 25173
E20260810 18:48:37.736976 25279 webserver.cc:533] Webserver: sslize failed:
E20260810 18:48:37.737071 25279 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-10T18:48:37Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (295 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-10T18:48:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:48:37Z Disabled control of system clock
I20260810 18:48:37.779181 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:37879
--webserver_interface=127.18.168.190
--webserver_port=0
--builtin_ntp_servers=127.18.168.148:34635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:37879
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387687857264-19106-0/testchainca.cert with env {}
W20260810 18:48:37.889407 25284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:37.889639 25284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:37.889684 25284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:37.893638 25284 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:48:37.893692 25284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:37.893705 25284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:37.893723 25284 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:48:37.893769 25284 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 18:48:37.893800 25284 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 18:48:37.893821 25284 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260810 18:48:37.898926 25284 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:34635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:37879
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:37879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387687857264-19106-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:37.899888 25284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:37.900909 25284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:37.909293 25293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:37.909317 25291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:37.909271 25290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:37.909586 25284 server_base.cc:1034] running on GCE node
I20260810 18:48:37.910156 25284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:37.911260 25284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:37.912454 25284 hybrid_clock.cc:648] HybridClock initialized: now 1786387717912442 us; error 27 us; skew 500 ppm
I20260810 18:48:37.914355 25284 webserver.cc:460] Webserver started at http://127.18.168.190:39861/ using document root <none> and password file <none>
I20260810 18:48:37.914937 25284 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:37.915027 25284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:37.915227 25284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:37.917284 25284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "bfda304b1c22437d80a5b5f8ec75bbcb"
format_stamp: "Formatted at 2026-08-10 18:48:37 on dist-test-slave-cbsf"
server_key: "b256e2e863e679aac83472172fc8b7f0"
server_key_iv: "9249c1be7c28afa4c2b3e567b6faf888"
server_key_version: "encryptionkey@0"
I20260810 18:48:37.917857 25284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "bfda304b1c22437d80a5b5f8ec75bbcb"
format_stamp: "Formatted at 2026-08-10 18:48:37 on dist-test-slave-cbsf"
server_key: "b256e2e863e679aac83472172fc8b7f0"
server_key_iv: "9249c1be7c28afa4c2b3e567b6faf888"
server_key_version: "encryptionkey@0"
I20260810 18:48:37.921445 25284 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:48:37.923857 25300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:37.925098 25284 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:48:37.925243 25284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "bfda304b1c22437d80a5b5f8ec75bbcb"
format_stamp: "Formatted at 2026-08-10 18:48:37 on dist-test-slave-cbsf"
server_key: "b256e2e863e679aac83472172fc8b7f0"
server_key_iv: "9249c1be7c28afa4c2b3e567b6faf888"
server_key_version: "encryptionkey@0"
I20260810 18:48:37.925351 25284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:37.941838 25284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:37.942466 25284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:37.942682 25284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:37.950182 25284 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:37879
I20260810 18:48:37.950212 25362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:37879 every 8 connection(s)
I20260810 18:48:37.951118 25284 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:48:37.954032 25363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:37.955024 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 25284
I20260810 18:48:37.955175 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:48:37.955462 19106 external_mini_cluster.cc:1428] Setting key 987cc8c249cc5380e21e583d05e29dda
I20260810 18:48:37.960428 25363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfda304b1c22437d80a5b5f8ec75bbcb: Bootstrap starting.
I20260810 18:48:37.962769 25363 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bfda304b1c22437d80a5b5f8ec75bbcb: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:37.963479 25363 log.cc:824] T 00000000000000000000000000000000 P bfda304b1c22437d80a5b5f8ec75bbcb: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:37.964804 25366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:37.956868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58226 (local address 127.18.168.190:37879)
0810 18:48:37.957528 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:37.957542 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:37.957576 (+ 34us) server_negotiation.cc:407] Connection header received
0810 18:48:37.958212 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:37.958229 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:37.958590 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:37.958925 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:37.959389 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:37.960348 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:37.961350 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:37.961827 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:37.962010 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:37.962896 (+ 886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:37.962903 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:37.962919 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:48:37.963297 (+ 378us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:37.963357 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:48:37.963377 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:37.963491 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:37.963655 (+ 164us) server_negotiation.cc:299] Negotiation successful
0810 18:48:37.963934 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":139,"threads_started":1}
I20260810 18:48:37.965566 25363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfda304b1c22437d80a5b5f8ec75bbcb: No bootstrap required, opened a new log
I20260810 18:48:37.967536 19106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:48:37.969753 25363 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfda304b1c22437d80a5b5f8ec75bbcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfda304b1c22437d80a5b5f8ec75bbcb" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 37879 } }
I20260810 18:48:37.969951 25363 raft_consensus.cc:374] T 00000000000000000000000000000000 P bfda304b1c22437d80a5b5f8ec75bbcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:37.969985 25363 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfda304b1c22437d80a5b5f8ec75bbcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfda304b1c22437d80a5b5f8ec75bbcb, State: Initialized, Role: FOLLOWER
I20260810 18:48:37.970326 25363 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfda304b1c22437d80a5b5f8ec75bbcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfda304b1c22437d80a5b5f8ec75bbcb" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 37879 } }
I20260810 18:48:37.970427 25363 raft_consensus.cc:388] T 00000000000000000000000000000000 P bfda304b1c22437d80a5b5f8ec75bbcb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:48:37.970467 25363 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfda304b1c22437d80a5b5f8ec75bbcb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:48:37.970542 25363 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfda304b1c22437d80a5b5f8ec75bbcb [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:37.971299 25363 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfda304b1c22437d80a5b5f8ec75bbcb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfda304b1c22437d80a5b5f8ec75bbcb" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 37879 } }
I20260810 18:48:37.971606 25363 leader_election.cc:302] T 00000000000000000000000000000000 P bfda304b1c22437d80a5b5f8ec75bbcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bfda304b1c22437d80a5b5f8ec75bbcb; no voters:
I20260810 18:48:37.971834 25363 leader_election.cc:288] T 00000000000000000000000000000000 P bfda304b1c22437d80a5b5f8ec75bbcb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:48:37.971956 25368 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfda304b1c22437d80a5b5f8ec75bbcb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:37.972232 25368 raft_consensus.cc:686] T 00000000000000000000000000000000 P bfda304b1c22437d80a5b5f8ec75bbcb [term 1 LEADER]: Becoming Leader. State: Replica: bfda304b1c22437d80a5b5f8ec75bbcb, State: Running, Role: LEADER
I20260810 18:48:37.972558 25368 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfda304b1c22437d80a5b5f8ec75bbcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfda304b1c22437d80a5b5f8ec75bbcb" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 37879 } }
I20260810 18:48:37.973037 25363 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfda304b1c22437d80a5b5f8ec75bbcb [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:48:37.974366 25369 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfda304b1c22437d80a5b5f8ec75bbcb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bfda304b1c22437d80a5b5f8ec75bbcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfda304b1c22437d80a5b5f8ec75bbcb" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 37879 } } }
I20260810 18:48:37.974406 25370 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfda304b1c22437d80a5b5f8ec75bbcb [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfda304b1c22437d80a5b5f8ec75bbcb. Latest consensus state: current_term: 1 leader_uuid: "bfda304b1c22437d80a5b5f8ec75bbcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfda304b1c22437d80a5b5f8ec75bbcb" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 37879 } } }
I20260810 18:48:37.974498 25369 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfda304b1c22437d80a5b5f8ec75bbcb [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:37.974509 25370 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfda304b1c22437d80a5b5f8ec75bbcb [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:37.975561 25378 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:48:37.978246 25378 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:48:37.983697 25378 catalog_manager.cc:1347] Generated new cluster ID: 33bcb94c8d5a435597982bdc3396f2b2
I20260810 18:48:37.983762 25378 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:48:38.003417 25378 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:48:38.004716 25378 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:48:38.014341 25378 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bfda304b1c22437d80a5b5f8ec75bbcb: Generated new TSK 0
I20260810 18:48:38.015123 25378 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:48:38.446094 25366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:38.027530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58230 (local address 127.18.168.190:37879)
0810 18:48:38.027663 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:38.027667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:38.027831 (+ 164us) server_negotiation.cc:407] Connection header received
0810 18:48:38.027960 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:38.027963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:38.028020 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:38.028138 (+ 118us) server_negotiation.cc:226] Negotiated authn=JWT
0810 18:48:38.028368 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:38.029250 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:38.030327 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:38.030674 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:38.030781 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:38.031057 (+ 276us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 18:48:38.445699 (+414642us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 18:48:38.445972 (+ 273us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58230: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":52}
W20260810 18:48:38.446228 25366 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58230: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260810 18:48:38.446322 25392 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:38.027412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:37879 (local address 127.0.0.1:58230)
0810 18:48:38.027729 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:38.027742 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:38.027855 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:38.028142 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:38.028144 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:38.028151 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0810 18:48:38.028262 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:38.028267 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:38.029376 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:38.029379 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:38.030183 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:38.030192 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:38.030793 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:38.030796 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:38.030863 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:38.030885 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 18:48:38.445957 (+415072us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 18:48:38.446138 (+ 181us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.168.190:37879: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":127,"threads_started":1}
I20260810 18:48:38.446978 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 25284
2026-08-10T18:48:38Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (699 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (124 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (123 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:48:38 dist-test-slave-cbsf krb5kdc[25410](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:48:38 dist-test-slave-cbsf krb5kdc[25410](info): set up 2 sockets
Aug 10 18:48:38 dist-test-slave-cbsf krb5kdc[25410](info): commencing operation
krb5kdc: starting...
W20260810 18:48:39.826747 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:48:39 dist-test-slave-cbsf krb5kdc[25410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:48:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:48:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.168.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
I20260810 18:48:39.951892 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:33391
--webserver_interface=127.18.168.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:44355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:33391
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:40.066416 25425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:40.066705 25425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:40.066776 25425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:40.070784 25425 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:48:40.070870 25425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:40.070909 25425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:40.070962 25425 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:48:40.070999 25425 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:48:40.076292 25425 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:44355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:33391
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:33391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:40.077420 25425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:40.078461 25425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:40.086972 25434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:40.086997 25436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:40.087013 25432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:40.087179 25425 server_base.cc:1034] running on GCE node
I20260810 18:48:40.087751 25425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:40.088896 25425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:40.090085 25425 hybrid_clock.cc:648] HybridClock initialized: now 1786387720090066 us; error 29 us; skew 500 ppm
Aug 10 18:48:40 dist-test-slave-cbsf krb5kdc[25410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387720, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:40.096825 25425 init.cc:377] Logged in from keytab as kudu/127.18.168.190@KRBTEST.COM (short username kudu)
I20260810 18:48:40.097693 25425 webserver.cc:460] Webserver started at http://127.18.168.190:34689/ using document root <none> and password file <none>
I20260810 18:48:40.098251 25425 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:40.098312 25425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:40.098475 25425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:40.100502 25425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "0e9cde406f684b78b8886bf983960852"
format_stamp: "Formatted at 2026-08-10 18:48:40 on dist-test-slave-cbsf"
server_key: "8af4af9ee4746eee1aa9ca2fbcce5973"
server_key_iv: "9704fad623237f121ea596ce1584a7ac"
server_key_version: "encryptionkey@0"
I20260810 18:48:40.101100 25425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "0e9cde406f684b78b8886bf983960852"
format_stamp: "Formatted at 2026-08-10 18:48:40 on dist-test-slave-cbsf"
server_key: "8af4af9ee4746eee1aa9ca2fbcce5973"
server_key_iv: "9704fad623237f121ea596ce1584a7ac"
server_key_version: "encryptionkey@0"
I20260810 18:48:40.104652 25425 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:48:40.107049 25443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:40.108240 25425 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:48:40.108342 25425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "0e9cde406f684b78b8886bf983960852"
format_stamp: "Formatted at 2026-08-10 18:48:40 on dist-test-slave-cbsf"
server_key: "8af4af9ee4746eee1aa9ca2fbcce5973"
server_key_iv: "9704fad623237f121ea596ce1584a7ac"
server_key_version: "encryptionkey@0"
I20260810 18:48:40.108423 25425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:40.121081 25425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:40.125113 25425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:40.125322 25425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:40.132120 25425 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:33391
I20260810 18:48:40.132198 25500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:33391 every 8 connection(s)
I20260810 18:48:40.133188 25425 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:48:40.136003 25501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:40.138252 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 25425
I20260810 18:48:40.138409 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:48:40.138692 19106 external_mini_cluster.cc:1428] Setting key a0de85b4ce5e44c43083e00596e47359
I20260810 18:48:40.142637 25501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e9cde406f684b78b8886bf983960852: Bootstrap starting.
I20260810 18:48:40.145334 25501 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e9cde406f684b78b8886bf983960852: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:40.146247 25501 log.cc:824] T 00000000000000000000000000000000 P 0e9cde406f684b78b8886bf983960852: Log is configured to *not* fsync() on all Append() calls
Aug 10 18:48:40 dist-test-slave-cbsf krb5kdc[25410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:40.148353 25501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e9cde406f684b78b8886bf983960852: No bootstrap required, opened a new log
I20260810 18:48:40.152046 25501 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e9cde406f684b78b8886bf983960852 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e9cde406f684b78b8886bf983960852" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 33391 } }
I20260810 18:48:40.152289 25501 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e9cde406f684b78b8886bf983960852 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:40.152374 25501 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e9cde406f684b78b8886bf983960852 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9cde406f684b78b8886bf983960852, State: Initialized, Role: FOLLOWER
I20260810 18:48:40.152941 25501 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e9cde406f684b78b8886bf983960852 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e9cde406f684b78b8886bf983960852" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 33391 } }
I20260810 18:48:40.153106 25501 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e9cde406f684b78b8886bf983960852 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:48:40.153172 25501 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e9cde406f684b78b8886bf983960852 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:48:40.153263 25501 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e9cde406f684b78b8886bf983960852 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:40.153900 25505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:40.140144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46388 (local address 127.18.168.190:33391)
0810 18:48:40.140584 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:40.140607 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:40.140662 (+ 55us) server_negotiation.cc:407] Connection header received
0810 18:48:40.141521 (+ 859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:40.141544 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:40.141968 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:40.142316 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:40.143076 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:40.144556 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:40.145485 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:40.146201 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:40.146372 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:40.148965 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:40.148970 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:40.148984 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:40.149009 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:40.150690 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:40.151135 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:40.151141 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:40.151149 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:40.151269 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:40.151558 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:40.151562 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:40.151563 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:40.151959 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:40.152183 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:40.152577 (+ 394us) server_negotiation.cc:299] Negotiation successful
0810 18:48:40.152907 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":112,"threads_started":1}
I20260810 18:48:40.154363 25501 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e9cde406f684b78b8886bf983960852 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e9cde406f684b78b8886bf983960852" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 33391 } }
I20260810 18:48:40.154734 25501 leader_election.cc:302] T 00000000000000000000000000000000 P 0e9cde406f684b78b8886bf983960852 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e9cde406f684b78b8886bf983960852; no voters:
I20260810 18:48:40.155072 25501 leader_election.cc:288] T 00000000000000000000000000000000 P 0e9cde406f684b78b8886bf983960852 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:48:40.155192 25509 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e9cde406f684b78b8886bf983960852 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:40.155412 25509 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e9cde406f684b78b8886bf983960852 [term 1 LEADER]: Becoming Leader. State: Replica: 0e9cde406f684b78b8886bf983960852, State: Running, Role: LEADER
I20260810 18:48:40.155771 25509 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e9cde406f684b78b8886bf983960852 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e9cde406f684b78b8886bf983960852" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 33391 } }
I20260810 18:48:40.156376 25501 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e9cde406f684b78b8886bf983960852 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:48:40.157608 25511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e9cde406f684b78b8886bf983960852 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e9cde406f684b78b8886bf983960852" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e9cde406f684b78b8886bf983960852" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 33391 } } }
I20260810 18:48:40.157744 25511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e9cde406f684b78b8886bf983960852 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:40.158133 25519 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:48:40.158104 25510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e9cde406f684b78b8886bf983960852 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e9cde406f684b78b8886bf983960852. Latest consensus state: current_term: 1 leader_uuid: "0e9cde406f684b78b8886bf983960852" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e9cde406f684b78b8886bf983960852" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 33391 } } }
I20260810 18:48:40.158272 25510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e9cde406f684b78b8886bf983960852 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:40.161020 25519 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:48:40.166931 25519 catalog_manager.cc:1347] Generated new cluster ID: 9ba85e49bbe940359db1112a38a29944
I20260810 18:48:40.166997 25519 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:48:40.178951 25519 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:48:40.180176 25519 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:48:40.187263 25519 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e9cde406f684b78b8886bf983960852: Generated new TSK 0
I20260810 18:48:40.187937 25519 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.168.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab.
Entry for principal kudu/127.18.168.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab.
I20260810 18:48:40.221550 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.129:0
--local_ip_for_outbound_sockets=127.18.168.129
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:33391
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:44355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:40.334452 25531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:40.334757 25531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:40.334846 25531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:40.338972 25531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:40.339056 25531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:40.339146 25531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.129
I20260810 18:48:40.344309 25531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:44355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:33391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.129
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:40.345377 25531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:40.346412 25531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:40.355819 25539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:40.355872 25531 server_base.cc:1034] running on GCE node
W20260810 18:48:40.355798 25537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:40.355819 25536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:40.356443 25531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:40.357188 25531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:40.358382 25531 hybrid_clock.cc:648] HybridClock initialized: now 1786387720358352 us; error 55 us; skew 500 ppm
Aug 10 18:48:40 dist-test-slave-cbsf krb5kdc[25410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387720, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:40.361255 25531 init.cc:377] Logged in from keytab as kudu/127.18.168.129@KRBTEST.COM (short username kudu)
I20260810 18:48:40.362125 25531 webserver.cc:460] Webserver started at http://127.18.168.129:33611/ using document root <none> and password file <none>
I20260810 18:48:40.362691 25531 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:40.362783 25531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:40.363022 25531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:40.365126 25531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-0/data/instance:
uuid: "cb3997c14474452caf68975c919a0ccc"
format_stamp: "Formatted at 2026-08-10 18:48:40 on dist-test-slave-cbsf"
server_key: "43c81f1bd45c0388a9cbc7939ca12c96"
server_key_iv: "6f4f75cf9d126b4897932c5cb22a16c2"
server_key_version: "encryptionkey@0"
I20260810 18:48:40.365696 25531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance:
uuid: "cb3997c14474452caf68975c919a0ccc"
format_stamp: "Formatted at 2026-08-10 18:48:40 on dist-test-slave-cbsf"
server_key: "43c81f1bd45c0388a9cbc7939ca12c96"
server_key_iv: "6f4f75cf9d126b4897932c5cb22a16c2"
server_key_version: "encryptionkey@0"
I20260810 18:48:40.369539 25531 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 18:48:40.371908 25547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:40.373102 25531 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 18:48:40.373232 25531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-0/wal
uuid: "cb3997c14474452caf68975c919a0ccc"
format_stamp: "Formatted at 2026-08-10 18:48:40 on dist-test-slave-cbsf"
server_key: "43c81f1bd45c0388a9cbc7939ca12c96"
server_key_iv: "6f4f75cf9d126b4897932c5cb22a16c2"
server_key_version: "encryptionkey@0"
I20260810 18:48:40.373338 25531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:40.404541 25531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:40.407822 25531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:40.408012 25531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:40.408597 25531 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:40.409392 25531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:40.409459 25531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:40.409547 25531 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:40.409591 25531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:40.418525 25531 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.129:44989
I20260810 18:48:40.418548 25672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.129:44989 every 8 connection(s)
I20260810 18:48:40.419337 25531 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
Aug 10 18:48:40 dist-test-slave-cbsf krb5kdc[25410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387720, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:40.428604 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 25531
I20260810 18:48:40.428756 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance
I20260810 18:48:40.429052 19106 external_mini_cluster.cc:1428] Setting key 69e23531fe7629a283e1edb9b68b06bc
I20260810 18:48:40.432608 25505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:40.421097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:40921 (local address 127.18.168.190:33391)
0810 18:48:40.421224 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:40.421228 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:40.422021 (+ 793us) server_negotiation.cc:407] Connection header received
0810 18:48:40.423370 (+ 1349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:40.423374 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:40.423433 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:40.423541 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:40.424856 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:40.425401 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:40.426524 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:40.426760 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:40.426869 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:40.429277 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:40.429280 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:40.429282 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:40.429305 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:40.430696 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:40.431161 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:40.431165 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:40.431167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:40.431243 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:40.431559 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:40.431563 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:40.431565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:40.431754 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:40.431861 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:40.432320 (+ 459us) server_negotiation.cc:299] Negotiation successful
0810 18:48:40.432441 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 18:48:40.433344 25675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:40.421432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:33391 (local address 127.18.168.129:40921)
0810 18:48:40.421883 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:40.421923 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:40.423164 (+ 1241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:40.423711 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:40.423718 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:40.424078 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:40.424655 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:40.424670 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:40.425546 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:40.425550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:40.426391 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:40.426400 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:40.426917 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:40.426921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:40.427082 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:40.427591 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:40.427606 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:40.429076 (+ 1470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:40.430810 (+ 1734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:40.430816 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:40.430828 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:40.431064 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:40.431329 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:40.431332 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:40.431334 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:40.431462 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:40.431861 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:40.431867 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:40.432096 (+ 229us) client_negotiation.cc:769] Sending connection context
0810 18:48:40.432275 (+ 179us) client_negotiation.cc:240] Negotiation successful
0810 18:48:40.432575 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":100,"threads_started":1}
I20260810 18:48:40.434494 25673 heartbeater.cc:347] Connected to a master server at 127.18.168.190:33391
I20260810 18:48:40.434760 25673 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:40.435360 25673 heartbeater.cc:511] Master 127.18.168.190:33391 requested a full tablet report, sending...
I20260810 18:48:40.437228 25461 ts_manager.cc:194] Registered new tserver with Master: cb3997c14474452caf68975c919a0ccc (127.18.168.129:44989)
I20260810 18:48:40.438952 25461 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.129@KRBTEST.COM'} at 127.18.168.129:40921
WARNING: no policy specified for kudu/127.18.168.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab.
Entry for principal kudu/127.18.168.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab.
I20260810 18:48:40.460002 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.130:0
--local_ip_for_outbound_sockets=127.18.168.130
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:33391
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:44355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:40.570266 25678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:40.570526 25678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:40.570595 25678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:40.574565 25678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:40.574636 25678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:40.574718 25678 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.130
I20260810 18:48:40.579843 25678 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:44355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:33391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.130
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:40.580914 25678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:40.581916 25678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:40.591296 25685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:40.591322 25687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:40.591338 25684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:40.591485 25678 server_base.cc:1034] running on GCE node
I20260810 18:48:40.591960 25678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:40.592562 25678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:40.593745 25678 hybrid_clock.cc:648] HybridClock initialized: now 1786387720593722 us; error 42 us; skew 500 ppm
Aug 10 18:48:40 dist-test-slave-cbsf krb5kdc[25410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387720, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:40.596508 25678 init.cc:377] Logged in from keytab as kudu/127.18.168.130@KRBTEST.COM (short username kudu)
I20260810 18:48:40.597371 25678 webserver.cc:460] Webserver started at http://127.18.168.130:46391/ using document root <none> and password file <none>
I20260810 18:48:40.597875 25678 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:40.597934 25678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:40.598093 25678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:40.599990 25678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-1/data/instance:
uuid: "9d607171a8b044219c8714b14b0f2fc8"
format_stamp: "Formatted at 2026-08-10 18:48:40 on dist-test-slave-cbsf"
server_key: "9022e506a0d3912bb1d462ecb8137de4"
server_key_iv: "13e8974a7d26c25ee437cdb439960b6b"
server_key_version: "encryptionkey@0"
I20260810 18:48:40.600487 25678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance:
uuid: "9d607171a8b044219c8714b14b0f2fc8"
format_stamp: "Formatted at 2026-08-10 18:48:40 on dist-test-slave-cbsf"
server_key: "9022e506a0d3912bb1d462ecb8137de4"
server_key_iv: "13e8974a7d26c25ee437cdb439960b6b"
server_key_version: "encryptionkey@0"
I20260810 18:48:40.604012 25678 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:48:40.606292 25694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:40.607478 25678 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:48:40.607594 25678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-1/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-1/wal
uuid: "9d607171a8b044219c8714b14b0f2fc8"
format_stamp: "Formatted at 2026-08-10 18:48:40 on dist-test-slave-cbsf"
server_key: "9022e506a0d3912bb1d462ecb8137de4"
server_key_iv: "13e8974a7d26c25ee437cdb439960b6b"
server_key_version: "encryptionkey@0"
I20260810 18:48:40.607692 25678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:40.629710 25678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:40.632398 25678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:40.632593 25678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:40.633188 25678 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:40.633980 25678 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:40.634027 25678 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:40.634090 25678 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:40.634130 25678 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:40.642733 25678 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.130:41387
I20260810 18:48:40.642755 25823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.130:41387 every 8 connection(s)
I20260810 18:48:40.643608 25678 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
I20260810 18:48:40.646538 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 25678
I20260810 18:48:40.646680 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance
I20260810 18:48:40.646960 19106 external_mini_cluster.cc:1428] Setting key ba08cf2c8af9bb019bfe48c6923957ce
Aug 10 18:48:40 dist-test-slave-cbsf krb5kdc[25410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387720, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:40.658178 25505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:40.645368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:46211 (local address 127.18.168.190:33391)
0810 18:48:40.645499 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:40.645503 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:40.646280 (+ 777us) server_negotiation.cc:407] Connection header received
0810 18:48:40.647779 (+ 1499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:40.647781 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:40.647823 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:40.647891 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:40.649163 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:40.649687 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:40.650853 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:40.651091 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:40.651207 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:40.654007 (+ 2800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:40.654010 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:40.654012 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:40.654034 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:40.655753 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:40.656254 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:40.656261 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:40.656263 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:40.656358 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:40.656807 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:40.656811 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:40.656815 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:40.657066 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:40.657197 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:40.657799 (+ 602us) server_negotiation.cc:299] Negotiation successful
0810 18:48:40.657937 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
WARNING: no policy specified for kudu/127.18.168.131@KRBTEST.COM; defaulting to no policy
I20260810 18:48:40.658958 25826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:40.645705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:33391 (local address 127.18.168.130:46211)
0810 18:48:40.646125 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:40.646167 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:40.647561 (+ 1394us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:40.648105 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:40.648113 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:40.648491 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:40.649013 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:40.649026 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:40.649828 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:40.649832 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:40.650721 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:40.650731 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:40.651267 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:40.651270 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:40.651445 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:40.651990 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:40.652006 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:40.653822 (+ 1816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:40.655875 (+ 2053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:40.655880 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:40.655903 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:40.656147 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:40.656522 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:40.656525 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:40.656527 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:40.656685 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:40.657199 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:40.657205 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:40.657471 (+ 266us) client_negotiation.cc:769] Sending connection context
0810 18:48:40.657699 (+ 228us) client_negotiation.cc:240] Negotiation successful
0810 18:48:40.657937 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":108,"threads_started":1}
I20260810 18:48:40.660120 25824 heartbeater.cc:347] Connected to a master server at 127.18.168.190:33391
I20260810 18:48:40.660363 25824 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:40.660959 25824 heartbeater.cc:511] Master 127.18.168.190:33391 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.131@KRBTEST.COM" created.
I20260810 18:48:40.662221 25461 ts_manager.cc:194] Registered new tserver with Master: 9d607171a8b044219c8714b14b0f2fc8 (127.18.168.130:41387)
I20260810 18:48:40.663071 25461 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.130@KRBTEST.COM'} at 127.18.168.130:46211
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab.
Entry for principal kudu/127.18.168.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab.
I20260810 18:48:40.675916 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.131:0
--local_ip_for_outbound_sockets=127.18.168.131
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:33391
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:44355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:40.788923 25829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:40.789204 25829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:40.789292 25829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:40.793565 25829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:40.793653 25829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:40.793741 25829 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.131
I20260810 18:48:40.798992 25829 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:44355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:33391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.131
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:40.800052 25829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:40.801177 25829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:40.810684 25837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:40.810822 25829 server_base.cc:1034] running on GCE node
W20260810 18:48:40.810735 25835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:40.810703 25834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:40.811452 25829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:40.812222 25829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:40.813410 25829 hybrid_clock.cc:648] HybridClock initialized: now 1786387720813389 us; error 31 us; skew 500 ppm
Aug 10 18:48:40 dist-test-slave-cbsf krb5kdc[25410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387720, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:40.816416 25829 init.cc:377] Logged in from keytab as kudu/127.18.168.131@KRBTEST.COM (short username kudu)
I20260810 18:48:40.817328 25829 webserver.cc:460] Webserver started at http://127.18.168.131:42121/ using document root <none> and password file <none>
I20260810 18:48:40.817929 25829 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:40.817998 25829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:40.818241 25829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:40.820228 25829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-2/data/instance:
uuid: "c7c7e83caaa443f085bd006bad922cc3"
format_stamp: "Formatted at 2026-08-10 18:48:40 on dist-test-slave-cbsf"
server_key: "9723404326a4601e0a5d7ab111c44638"
server_key_iv: "b4b26ba35e5a408a5a0d680dcd96daeb"
server_key_version: "encryptionkey@0"
I20260810 18:48:40.820856 25829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance:
uuid: "c7c7e83caaa443f085bd006bad922cc3"
format_stamp: "Formatted at 2026-08-10 18:48:40 on dist-test-slave-cbsf"
server_key: "9723404326a4601e0a5d7ab111c44638"
server_key_iv: "b4b26ba35e5a408a5a0d680dcd96daeb"
server_key_version: "encryptionkey@0"
I20260810 18:48:40.824465 25829 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:48:40.827003 25844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:40.828131 25829 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:48:40.828250 25829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-2/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-2/wal
uuid: "c7c7e83caaa443f085bd006bad922cc3"
format_stamp: "Formatted at 2026-08-10 18:48:40 on dist-test-slave-cbsf"
server_key: "9723404326a4601e0a5d7ab111c44638"
server_key_iv: "b4b26ba35e5a408a5a0d680dcd96daeb"
server_key_version: "encryptionkey@0"
I20260810 18:48:40.828346 25829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:40.846895 25829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:40.850169 25829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:40.850322 25829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:40.850931 25829 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:40.851830 25829 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:40.851878 25829 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:40.851943 25829 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:40.851989 25829 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:40.861698 25829 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.131:44553
I20260810 18:48:40.861722 25972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.131:44553 every 8 connection(s)
I20260810 18:48:40.862600 25829 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
Aug 10 18:48:40 dist-test-slave-cbsf krb5kdc[25410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387720, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:40.872260 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 25829
I20260810 18:48:40.872396 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance
I20260810 18:48:40.872826 19106 external_mini_cluster.cc:1428] Setting key bd096a690c8e4a342077509b3bee6c12
I20260810 18:48:40.876354 25505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:40.864409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:37341 (local address 127.18.168.190:33391)
0810 18:48:40.864567 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:40.864571 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:40.865510 (+ 939us) server_negotiation.cc:407] Connection header received
0810 18:48:40.866819 (+ 1309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:40.866821 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:40.866869 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:40.866944 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:40.868035 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:40.868608 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:40.869799 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:40.870022 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:40.870109 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:40.872654 (+ 2545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:40.872658 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:40.872660 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:40.872686 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:40.874421 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:40.874941 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:40.874945 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:40.874947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:40.875047 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:40.875397 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:40.875400 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:40.875401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:40.875575 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:40.875654 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:40.876134 (+ 480us) server_negotiation.cc:299] Negotiation successful
0810 18:48:40.876239 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 18:48:40.877211 25975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:40.864813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:33391 (local address 127.18.168.131:37341)
0810 18:48:40.865358 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:40.865400 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:40.866620 (+ 1220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:40.867106 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:40.867112 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:40.867443 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:40.867889 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:40.867901 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:40.868774 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:40.868783 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:40.869680 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:40.869690 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:40.870165 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:40.870168 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:40.870316 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:40.870836 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:40.870851 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:40.872478 (+ 1627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:40.874535 (+ 2057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:40.874540 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:40.874550 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:40.874816 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:40.875148 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:40.875151 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:40.875153 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:40.875309 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:40.875664 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:40.875669 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:40.875881 (+ 212us) client_negotiation.cc:769] Sending connection context
0810 18:48:40.876066 (+ 185us) client_negotiation.cc:240] Negotiation successful
0810 18:48:40.876315 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":167,"threads_started":1}
I20260810 18:48:40.878319 25973 heartbeater.cc:347] Connected to a master server at 127.18.168.190:33391
I20260810 18:48:40.878600 25973 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:40.879087 25973 heartbeater.cc:511] Master 127.18.168.190:33391 requested a full tablet report, sending...
I20260810 18:48:40.880106 25461 ts_manager.cc:194] Registered new tserver with Master: c7c7e83caaa443f085bd006bad922cc3 (127.18.168.131:44553)
I20260810 18:48:40.880640 25461 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.131@KRBTEST.COM'} at 127.18.168.131:37341
I20260810 18:48:40.886680 19106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/krb5kdc/test-admin.keytab.
Aug 10 18:48:40 dist-test-slave-cbsf krb5kdc[25410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:40.903156 19106 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260810 18:48:40.905957 25980 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:40.904503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60300 (local address 127.18.168.129:44989)
0810 18:48:40.904853 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:40.904877 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:40.904901 (+ 24us) server_negotiation.cc:407] Connection header received
0810 18:48:40.904960 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:40.904965 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:40.905141 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:40.905293 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:40.905770 (+ 477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60300: BlockingRecv error: recv got EOF from 127.0.0.1:60300 (error 108)
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":86,"threads_started":1}
W20260810 18:48:40.906064 19106 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/krb5kdc/krb5cc' of type FILE
Aug 10 18:48:40 dist-test-slave-cbsf krb5kdc[25410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:40.907130 19106 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 18:48:40 dist-test-slave-cbsf krb5kdc[25410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:40.908093 19106 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 18:48:40 dist-test-slave-cbsf krb5kdc[25410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.168.129@KRBTEST.COM
I20260810 18:48:40.918109 25980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:40.908955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60314 (local address 127.18.168.129:44989)
0810 18:48:40.909105 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:40.909110 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:40.909231 (+ 121us) server_negotiation.cc:407] Connection header received
0810 18:48:40.909380 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:40.909383 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:40.909432 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:40.909517 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:40.910050 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:40.910954 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:40.911983 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:40.912440 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:40.912558 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:40.914837 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:40.914847 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:40.914861 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:40.914884 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:40.916306 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:40.916838 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:40.916851 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:40.916853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:40.916939 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:40.917196 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:40.917199 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:40.917200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:40.917432 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:40.917602 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:40.917782 (+ 180us) server_negotiation.cc:299] Negotiation successful
0810 18:48:40.917971 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260810 18:48:40.922067 25980 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:40.920906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60330 (local address 127.18.168.129:44989)
0810 18:48:40.921046 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:40.921050 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:40.921173 (+ 123us) server_negotiation.cc:407] Connection header received
0810 18:48:40.921318 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:40.921321 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:40.921360 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:40.921446 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:40.921950 (+ 504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60330: BlockingRecv error: recv got EOF from 127.0.0.1:60330 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260810 18:48:40.922405 19106 init.cc:244] Error while opening credential cache '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/krb5kdc/krb5cc' of type FILE: Runtime error: Bad format in credentials cache (filename: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/krb5kdc/krb5cc)
Aug 10 18:48:40 dist-test-slave-cbsf krb5kdc[25410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:40.923418 19106 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 18:48:40 dist-test-slave-cbsf krb5kdc[25410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:40.924297 19106 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 18:48:40 dist-test-slave-cbsf krb5kdc[25410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.168.129@KRBTEST.COM
I20260810 18:48:40.932430 25980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:40.925127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60334 (local address 127.18.168.129:44989)
0810 18:48:40.925275 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:40.925278 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:40.925341 (+ 63us) server_negotiation.cc:407] Connection header received
0810 18:48:40.925471 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:40.925473 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:40.925506 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:40.925562 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:40.926019 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:40.926493 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:40.927201 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:40.927410 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:40.927481 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:40.929667 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:40.929669 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:40.929671 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:40.929691 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:40.930871 (+ 1180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:40.931277 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:40.931281 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:40.931285 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:40.931368 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:40.931599 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:40.931601 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:40.931602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:40.931750 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:40.931898 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:40.932114 (+ 216us) server_negotiation.cc:299] Negotiation successful
0810 18:48:40.932237 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260810 18:48:40.934979 25980 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:40.934233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60340 (local address 127.18.168.129:44989)
0810 18:48:40.934334 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:40.934337 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:40.934416 (+ 79us) server_negotiation.cc:407] Connection header received
0810 18:48:40.934492 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:40.934494 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:40.934526 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:40.934587 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:40.934869 (+ 282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60340: BlockingRecv error: recv got EOF from 127.0.0.1:60340 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260810 18:48:40.935104 19106 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387687857264-19106-0/krb5kdc/krb5cc' of type FILE
Aug 10 18:48:40 dist-test-slave-cbsf krb5kdc[25410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:40.936007 19106 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 18:48:40 dist-test-slave-cbsf krb5kdc[25410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:40.936910 19106 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 18:48:40 dist-test-slave-cbsf krb5kdc[25410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.168.129@KRBTEST.COM
I20260810 18:48:40.945508 25980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:40.937663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60348 (local address 127.18.168.129:44989)
0810 18:48:40.937783 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:40.937787 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:40.937905 (+ 118us) server_negotiation.cc:407] Connection header received
0810 18:48:40.938055 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:40.938057 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:40.938090 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:40.938181 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:40.938652 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:40.939134 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:40.939915 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:40.940122 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:40.940235 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:40.942265 (+ 2030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:40.942271 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:40.942278 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:40.942299 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:40.943926 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:40.944338 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:40.944342 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:40.944343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:40.944428 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:40.944729 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:40.944734 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:40.944736 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:40.944898 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:40.945027 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:40.945260 (+ 233us) server_negotiation.cc:299] Negotiation successful
0810 18:48:40.945361 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 18:48:40.946415 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 25531
I20260810 18:48:40.955751 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 25678
I20260810 18:48:40.964403 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 25829
I20260810 18:48:40.971709 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 25425
2026-08-10T18:48:40Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2280 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:48:41 dist-test-slave-cbsf krb5kdc[25994](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:48:41 dist-test-slave-cbsf krb5kdc[25994](info): set up 2 sockets
Aug 10 18:48:41 dist-test-slave-cbsf krb5kdc[25994](info): commencing operation
krb5kdc: starting...
W20260810 18:48:42.127529 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:48:42 dist-test-slave-cbsf krb5kdc[25994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387722, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:48:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:48:42Z Disabled control of system clock
WARNING: no policy specified for oryx/127.18.168.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.168.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.168.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.190.keytab.
Entry for principal oryx/127.18.168.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.190.keytab.
I20260810 18:48:42.255276 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:43053
--webserver_interface=127.18.168.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.190.keytab
--principal=oryx/127.18.168.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:37727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:43053
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:42.364459 26010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:42.364781 26010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:42.364835 26010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:42.368973 26010 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:48:42.369030 26010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:42.369050 26010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:42.369066 26010 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:48:42.369081 26010 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:48:42.374320 26010 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:37727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:43053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.190.keytab
--principal=oryx/127.18.168.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:43053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26010
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:42.375264 26010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:42.376204 26010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:42.384737 26017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:42.384841 26019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:42.384789 26016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:42.385069 26010 server_base.cc:1034] running on GCE node
I20260810 18:48:42.385581 26010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:42.386617 26010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:42.387789 26010 hybrid_clock.cc:648] HybridClock initialized: now 1786387722387761 us; error 38 us; skew 500 ppm
Aug 10 18:48:42 dist-test-slave-cbsf krb5kdc[25994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387722, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.168.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:42.390600 26010 init.cc:377] Logged in from keytab as oryx/127.18.168.190@KRBTEST.COM (short username oryx)
I20260810 18:48:42.391489 26010 webserver.cc:460] Webserver started at http://127.18.168.190:45177/ using document root <none> and password file <none>
I20260810 18:48:42.392040 26010 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:42.392132 26010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:42.392385 26010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:42.394438 26010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "1dde94f763cc421f8e7b1693c060e3f3"
format_stamp: "Formatted at 2026-08-10 18:48:42 on dist-test-slave-cbsf"
server_key: "9c8ea042787e8b13a08be1826a2335be"
server_key_iv: "52029ab9677739620242e4fe86e190b4"
server_key_version: "encryptionkey@0"
I20260810 18:48:42.395021 26010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "1dde94f763cc421f8e7b1693c060e3f3"
format_stamp: "Formatted at 2026-08-10 18:48:42 on dist-test-slave-cbsf"
server_key: "9c8ea042787e8b13a08be1826a2335be"
server_key_iv: "52029ab9677739620242e4fe86e190b4"
server_key_version: "encryptionkey@0"
I20260810 18:48:42.398706 26010 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 18:48:42.401085 26028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:42.402262 26010 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:48:42.402403 26010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "1dde94f763cc421f8e7b1693c060e3f3"
format_stamp: "Formatted at 2026-08-10 18:48:42 on dist-test-slave-cbsf"
server_key: "9c8ea042787e8b13a08be1826a2335be"
server_key_iv: "52029ab9677739620242e4fe86e190b4"
server_key_version: "encryptionkey@0"
I20260810 18:48:42.402508 26010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:42.427301 26010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:42.430303 26010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:42.430509 26010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:42.437629 26010 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:43053
I20260810 18:48:42.437894 26084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:43053 every 8 connection(s)
I20260810 18:48:42.438620 26010 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:48:42.441308 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 26010
I20260810 18:48:42.441435 26085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:42.441462 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:48:42.441790 19106 external_mini_cluster.cc:1428] Setting key b6a48a685254a1398aa1cba840091f94
I20260810 18:48:42.448169 26085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3: Bootstrap starting.
I20260810 18:48:42.450989 26085 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3: Neither blocks nor log segments found. Creating new log.
Aug 10 18:48:42 dist-test-slave-cbsf krb5kdc[25994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387722, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM
I20260810 18:48:42.451911 26085 log.cc:824] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:42.454113 26085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3: No bootstrap required, opened a new log
I20260810 18:48:42.456457 26088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:42.443122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52438 (local address 127.18.168.190:43053)
0810 18:48:42.443545 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:42.443559 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:42.443594 (+ 35us) server_negotiation.cc:407] Connection header received
0810 18:48:42.444552 (+ 958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:42.444571 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:42.444954 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:42.445230 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:42.446166 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:42.447084 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:42.448191 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:42.448795 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:42.448920 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:42.451699 (+ 2779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:42.451705 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:42.451717 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:42.451743 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:42.453603 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:42.454102 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:42.454107 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:42.454112 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:42.454284 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:42.454598 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:42.454600 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:42.454601 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:42.454994 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:42.455129 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:42.455449 (+ 320us) server_negotiation.cc:299] Negotiation successful
0810 18:48:42.455755 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":129,"threads_started":1}
I20260810 18:48:42.458057 26085 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } }
I20260810 18:48:42.458292 26085 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:42.458381 26085 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dde94f763cc421f8e7b1693c060e3f3, State: Initialized, Role: FOLLOWER
I20260810 18:48:42.458817 26085 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } }
I20260810 18:48:42.458942 26085 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:48:42.458984 26085 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:48:42.459055 26085 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:42.459878 26085 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } }
I20260810 18:48:42.460166 26085 leader_election.cc:302] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1dde94f763cc421f8e7b1693c060e3f3; no voters:
I20260810 18:48:42.460417 26085 leader_election.cc:288] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:48:42.460557 26090 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:42.460877 26090 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [term 1 LEADER]: Becoming Leader. State: Replica: 1dde94f763cc421f8e7b1693c060e3f3, State: Running, Role: LEADER
I20260810 18:48:42.461294 26090 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } }
I20260810 18:48:42.461678 26085 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:48:42.462888 26092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1dde94f763cc421f8e7b1693c060e3f3. Latest consensus state: current_term: 1 leader_uuid: "1dde94f763cc421f8e7b1693c060e3f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } }
I20260810 18:48:42.463024 26092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:42.463462 26100 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:48:42.463414 26091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1dde94f763cc421f8e7b1693c060e3f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } }
I20260810 18:48:42.463634 26091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:42.466755 26100 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:48:42.472136 26100 catalog_manager.cc:1347] Generated new cluster ID: 4f874a85450b418499bc2b5946eac0c8
I20260810 18:48:42.472203 26100 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:48:42.477936 26100 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:48:42.478781 26100 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:48:42.486126 26100 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3: Generated new TSK 0
I20260810 18:48:42.486773 26100 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.18.168.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.168.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.168.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.129.keytab.
Entry for principal oryx/127.18.168.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.129.keytab.
I20260810 18:48:42.521777 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.129:0
--local_ip_for_outbound_sockets=127.18.168.129
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:43053
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.129.keytab
--principal=oryx/127.18.168.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:37727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:42.636159 26115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:42.636432 26115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:42.636531 26115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:42.640664 26115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:42.640741 26115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:42.640832 26115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.129
I20260810 18:48:42.645954 26115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:37727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.129.keytab
--principal=oryx/127.18.168.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:43053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.129
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:42.646930 26115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:42.648011 26115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:42.657228 26121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:42.657251 26120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:42.657213 26123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:42.657538 26115 server_base.cc:1034] running on GCE node
I20260810 18:48:42.657869 26115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:42.658510 26115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:42.659679 26115 hybrid_clock.cc:648] HybridClock initialized: now 1786387722659666 us; error 24 us; skew 500 ppm
Aug 10 18:48:42 dist-test-slave-cbsf krb5kdc[25994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387722, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.168.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:42.662684 26115 init.cc:377] Logged in from keytab as oryx/127.18.168.129@KRBTEST.COM (short username oryx)
I20260810 18:48:42.663556 26115 webserver.cc:460] Webserver started at http://127.18.168.129:44363/ using document root <none> and password file <none>
I20260810 18:48:42.664049 26115 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:42.664106 26115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:42.664268 26115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:42.666254 26115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-0/data/instance:
uuid: "7023e9dd2aaa481cbe00bfe83efac6ed"
format_stamp: "Formatted at 2026-08-10 18:48:42 on dist-test-slave-cbsf"
server_key: "3815e100d495fa3444dab51b3d7b7a7c"
server_key_iv: "5b6222a5219b727cd1ea9ac29f9f45d2"
server_key_version: "encryptionkey@0"
I20260810 18:48:42.666776 26115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance:
uuid: "7023e9dd2aaa481cbe00bfe83efac6ed"
format_stamp: "Formatted at 2026-08-10 18:48:42 on dist-test-slave-cbsf"
server_key: "3815e100d495fa3444dab51b3d7b7a7c"
server_key_iv: "5b6222a5219b727cd1ea9ac29f9f45d2"
server_key_version: "encryptionkey@0"
I20260810 18:48:42.670600 26115 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:48:42.673007 26131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:42.674253 26115 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 18:48:42.674376 26115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-0/wal
uuid: "7023e9dd2aaa481cbe00bfe83efac6ed"
format_stamp: "Formatted at 2026-08-10 18:48:42 on dist-test-slave-cbsf"
server_key: "3815e100d495fa3444dab51b3d7b7a7c"
server_key_iv: "5b6222a5219b727cd1ea9ac29f9f45d2"
server_key_version: "encryptionkey@0"
I20260810 18:48:42.674551 26115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:42.698851 26115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:42.702031 26115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:42.702243 26115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:42.702872 26115 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:42.703732 26115 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:42.703780 26115 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:42.703820 26115 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:42.703877 26115 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:42.713500 26115 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.129:42891
I20260810 18:48:42.713521 26259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.129:42891 every 8 connection(s)
I20260810 18:48:42.714370 26115 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
I20260810 18:48:42.719180 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 26115
I20260810 18:48:42.719338 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance
I20260810 18:48:42.719750 19106 external_mini_cluster.cc:1428] Setting key 123fcb2afebfd01e6ef09f3117515056
Aug 10 18:48:42 dist-test-slave-cbsf krb5kdc[25994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387722, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.168.129@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM
I20260810 18:48:42.731123 26088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:42.716212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:35883 (local address 127.18.168.190:43053)
0810 18:48:42.716386 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:42.716390 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:42.717258 (+ 868us) server_negotiation.cc:407] Connection header received
0810 18:48:42.718721 (+ 1463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:42.718724 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:42.718771 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:42.718846 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:42.720594 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:42.721240 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:42.723108 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:42.723337 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:42.723422 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:42.726553 (+ 3131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:42.726560 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:42.726566 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:42.726603 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:42.728667 (+ 2064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:42.729232 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:42.729238 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:42.729241 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:42.729345 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:42.729777 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:42.729784 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:42.729787 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:42.730027 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:42.730145 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:42.730772 (+ 627us) server_negotiation.cc:299] Negotiation successful
0810 18:48:42.730916 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
WARNING: no policy specified for oryx/127.18.168.130@KRBTEST.COM; defaulting to no policy
I20260810 18:48:42.732041 26262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:42.716530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43053 (local address 127.18.168.129:35883)
0810 18:48:42.717080 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:42.717126 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:42.718517 (+ 1391us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:42.719171 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:42.719182 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:42.719722 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:42.720393 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:42.720435 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:42.721389 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:42.721393 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:42.722966 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:42.722982 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:42.723502 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:42.723510 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:42.723760 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:42.724518 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:42.724535 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:42.726351 (+ 1816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:42.728830 (+ 2479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:42.728835 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:42.728847 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:42.729123 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:42.729503 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:42.729507 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:42.729509 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:42.729663 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:42.730175 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:42.730181 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:42.730487 (+ 306us) client_negotiation.cc:769] Sending connection context
0810 18:48:42.730681 (+ 194us) client_negotiation.cc:240] Negotiation successful
0810 18:48:42.730978 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":192,"threads_started":1}
I20260810 18:48:42.733404 26260 heartbeater.cc:347] Connected to a master server at 127.18.168.190:43053
I20260810 18:48:42.733640 26260 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:42.734143 26260 heartbeater.cc:511] Master 127.18.168.190:43053 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.168.130@KRBTEST.COM" created.
I20260810 18:48:42.735905 26046 ts_manager.cc:194] Registered new tserver with Master: 7023e9dd2aaa481cbe00bfe83efac6ed (127.18.168.129:42891)
I20260810 18:48:42.737213 26046 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.168.129@KRBTEST.COM'} at 127.18.168.129:35883
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.168.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.130.keytab.
Entry for principal oryx/127.18.168.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.130.keytab.
I20260810 18:48:42.752313 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.130:0
--local_ip_for_outbound_sockets=127.18.168.130
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:43053
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.130.keytab
--principal=oryx/127.18.168.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:37727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:42.864048 26265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:42.864313 26265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:42.864355 26265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:42.868322 26265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:42.868376 26265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:42.868431 26265 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.130
I20260810 18:48:42.873549 26265 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:37727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.130.keytab
--principal=oryx/127.18.168.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:43053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.130
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:42.874502 26265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:42.875469 26265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:42.885085 26272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:42.885053 26274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:42.885212 26271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:42.885331 26265 server_base.cc:1034] running on GCE node
I20260810 18:48:42.885838 26265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:42.886479 26265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:42.887667 26265 hybrid_clock.cc:648] HybridClock initialized: now 1786387722887655 us; error 24 us; skew 500 ppm
Aug 10 18:48:42 dist-test-slave-cbsf krb5kdc[25994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387722, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.168.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:42.890558 26265 init.cc:377] Logged in from keytab as oryx/127.18.168.130@KRBTEST.COM (short username oryx)
I20260810 18:48:42.891471 26265 webserver.cc:460] Webserver started at http://127.18.168.130:39437/ using document root <none> and password file <none>
I20260810 18:48:42.892050 26265 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:42.892175 26265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:42.892416 26265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:42.894516 26265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-1/data/instance:
uuid: "59c9f2d08b94401989fdc32d65539feb"
format_stamp: "Formatted at 2026-08-10 18:48:42 on dist-test-slave-cbsf"
server_key: "420784fd2fb49bd4348fba68df1008aa"
server_key_iv: "bded2997401be0ae2150ea84d0183628"
server_key_version: "encryptionkey@0"
I20260810 18:48:42.895095 26265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance:
uuid: "59c9f2d08b94401989fdc32d65539feb"
format_stamp: "Formatted at 2026-08-10 18:48:42 on dist-test-slave-cbsf"
server_key: "420784fd2fb49bd4348fba68df1008aa"
server_key_iv: "bded2997401be0ae2150ea84d0183628"
server_key_version: "encryptionkey@0"
I20260810 18:48:42.898932 26265 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:48:42.901443 26282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:42.902673 26265 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:48:42.902810 26265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-1/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-1/wal
uuid: "59c9f2d08b94401989fdc32d65539feb"
format_stamp: "Formatted at 2026-08-10 18:48:42 on dist-test-slave-cbsf"
server_key: "420784fd2fb49bd4348fba68df1008aa"
server_key_iv: "bded2997401be0ae2150ea84d0183628"
server_key_version: "encryptionkey@0"
I20260810 18:48:42.902917 26265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:42.920776 26265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:42.923883 26265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:42.924098 26265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:42.924710 26265 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:42.925546 26265 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:42.925618 26265 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:42.925694 26265 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:42.925746 26265 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:42.934788 26265 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.130:41981
I20260810 18:48:42.934823 26412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.130:41981 every 8 connection(s)
I20260810 18:48:42.935642 26265 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
I20260810 18:48:42.939183 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 26265
I20260810 18:48:42.939335 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance
I20260810 18:48:42.939636 19106 external_mini_cluster.cc:1428] Setting key 682daed7059eb1fe1ea59042f53a2280
Aug 10 18:48:42 dist-test-slave-cbsf krb5kdc[25994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387722, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.168.130@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.18.168.131@KRBTEST.COM; defaulting to no policy
I20260810 18:48:42.951057 26088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:42.937466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:49969 (local address 127.18.168.190:43053)
0810 18:48:42.937606 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:42.937610 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:42.938422 (+ 812us) server_negotiation.cc:407] Connection header received
0810 18:48:42.940043 (+ 1621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:42.940046 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:42.940092 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:42.940164 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:42.941667 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:42.942509 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:42.943917 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:42.944131 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:42.944208 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:42.947291 (+ 3083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:42.947296 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:42.947299 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:42.947328 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:42.948963 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:42.949499 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:42.949507 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:42.949509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:42.949584 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:42.949926 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:42.949930 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:42.949932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:42.950117 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:42.950227 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:42.950735 (+ 508us) server_negotiation.cc:299] Negotiation successful
0810 18:48:42.950852 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 18:48:42.951941 26415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:42.937814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43053 (local address 127.18.168.130:49969)
0810 18:48:42.938257 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:42.938298 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:42.939810 (+ 1512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:42.940358 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:42.940365 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:42.940915 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:42.941501 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:42.941534 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:42.942633 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:42.942636 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:42.943785 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:42.943800 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:42.944278 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:42.944282 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:42.944500 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:42.945202 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:42.945218 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:42.947071 (+ 1853us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:42.949099 (+ 2028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:42.949105 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:42.949120 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:42.949378 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:42.949677 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:42.949682 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:42.949684 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:42.949824 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:42.950226 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:42.950233 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:42.950479 (+ 246us) client_negotiation.cc:769] Sending connection context
0810 18:48:42.950688 (+ 209us) client_negotiation.cc:240] Negotiation successful
0810 18:48:42.951034 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":117,"threads_started":1}
I20260810 18:48:42.953128 26413 heartbeater.cc:347] Connected to a master server at 127.18.168.190:43053
I20260810 18:48:42.953406 26413 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.168.131@KRBTEST.COM" created.
I20260810 18:48:42.953995 26413 heartbeater.cc:511] Master 127.18.168.190:43053 requested a full tablet report, sending...
I20260810 18:48:42.955180 26046 ts_manager.cc:194] Registered new tserver with Master: 59c9f2d08b94401989fdc32d65539feb (127.18.168.130:41981)
I20260810 18:48:42.955829 26046 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.168.130@KRBTEST.COM'} at 127.18.168.130:49969
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.168.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.131.keytab.
Entry for principal oryx/127.18.168.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.131.keytab.
I20260810 18:48:42.972556 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.131:0
--local_ip_for_outbound_sockets=127.18.168.131
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:43053
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.131.keytab
--principal=oryx/127.18.168.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:37727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:43.096382 26418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:43.096655 26418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:43.096761 26418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:43.100871 26418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:43.100921 26418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:43.100978 26418 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.131
I20260810 18:48:43.106151 26418 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:37727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.131.keytab
--principal=oryx/127.18.168.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:43053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.131
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:43.107200 26418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:43.108155 26418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:43.117937 26424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:43.117933 26427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:43.117937 26423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:43.117976 26418 server_base.cc:1034] running on GCE node
I20260810 18:48:43.118587 26418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:43.119225 26418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:43.120424 26418 hybrid_clock.cc:648] HybridClock initialized: now 1786387723120390 us; error 46 us; skew 500 ppm
Aug 10 18:48:43 dist-test-slave-cbsf krb5kdc[25994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387723, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.168.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:43.123210 26418 init.cc:377] Logged in from keytab as oryx/127.18.168.131@KRBTEST.COM (short username oryx)
I20260810 18:48:43.124130 26418 webserver.cc:460] Webserver started at http://127.18.168.131:45665/ using document root <none> and password file <none>
I20260810 18:48:43.124737 26418 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:43.124841 26418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:43.125025 26418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:43.127080 26418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-2/data/instance:
uuid: "aca3a4b0c4f8484c9a62e05683ce453e"
format_stamp: "Formatted at 2026-08-10 18:48:43 on dist-test-slave-cbsf"
server_key: "7b252ff7076d56ae61b11ca383ed0457"
server_key_iv: "99a0453cf8c07041c6587739ff0858df"
server_key_version: "encryptionkey@0"
I20260810 18:48:43.127624 26418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance:
uuid: "aca3a4b0c4f8484c9a62e05683ce453e"
format_stamp: "Formatted at 2026-08-10 18:48:43 on dist-test-slave-cbsf"
server_key: "7b252ff7076d56ae61b11ca383ed0457"
server_key_iv: "99a0453cf8c07041c6587739ff0858df"
server_key_version: "encryptionkey@0"
I20260810 18:48:43.131634 26418 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:48:43.134610 26435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:43.136114 26418 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 18:48:43.136245 26418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-2/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-2/wal
uuid: "aca3a4b0c4f8484c9a62e05683ce453e"
format_stamp: "Formatted at 2026-08-10 18:48:43 on dist-test-slave-cbsf"
server_key: "7b252ff7076d56ae61b11ca383ed0457"
server_key_iv: "99a0453cf8c07041c6587739ff0858df"
server_key_version: "encryptionkey@0"
I20260810 18:48:43.136361 26418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:43.159765 26418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:43.162791 26418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:43.162951 26418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:43.163558 26418 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:43.164414 26418 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:43.164461 26418 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:43.164529 26418 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:43.164561 26418 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:43.173863 26418 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.131:45911
I20260810 18:48:43.173893 26563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.131:45911 every 8 connection(s)
I20260810 18:48:43.174767 26418 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
I20260810 18:48:43.179435 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 26418
I20260810 18:48:43.179562 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance
I20260810 18:48:43.179863 19106 external_mini_cluster.cc:1428] Setting key 510f05dd2d477c844b9b3689a9c72e7d
Aug 10 18:48:43 dist-test-slave-cbsf krb5kdc[25994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387723, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.168.131@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM
I20260810 18:48:43.188504 26088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:43.176598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:46953 (local address 127.18.168.190:43053)
0810 18:48:43.176762 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:43.176766 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:43.177590 (+ 824us) server_negotiation.cc:407] Connection header received
0810 18:48:43.179081 (+ 1491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:43.179085 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:43.179147 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:43.179252 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:43.180616 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.181360 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.182631 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.182880 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.182984 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:43.185404 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:43.185407 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:43.185409 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:43.185431 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:43.186729 (+ 1298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:43.187189 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:43.187191 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:43.187192 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:43.187270 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:43.187570 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:43.187572 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:43.187573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:43.187734 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:43.187806 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:43.188278 (+ 472us) server_negotiation.cc:299] Negotiation successful
0810 18:48:43.188387 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 18:48:43.189297 26566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:43.176960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43053 (local address 127.18.168.131:46953)
0810 18:48:43.177431 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:43.177473 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:43.178858 (+ 1385us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:43.179427 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:43.179433 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:43.179878 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:43.180456 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:43.180477 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.181480 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.181484 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:43.182494 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:43.182505 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.183039 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.183042 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:43.183200 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:43.183723 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:43.183740 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:43.185244 (+ 1504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:43.186841 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:43.186846 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:43.186857 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:43.187101 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:43.187350 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:43.187353 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:43.187354 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:43.187479 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:43.187818 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:43.187823 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:43.188073 (+ 250us) client_negotiation.cc:769] Sending connection context
0810 18:48:43.188232 (+ 159us) client_negotiation.cc:240] Negotiation successful
0810 18:48:43.188482 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":129,"threads_started":1}
I20260810 18:48:43.190373 26564 heartbeater.cc:347] Connected to a master server at 127.18.168.190:43053
I20260810 18:48:43.190577 26564 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:43.191015 26564 heartbeater.cc:511] Master 127.18.168.190:43053 requested a full tablet report, sending...
I20260810 18:48:43.191992 26046 ts_manager.cc:194] Registered new tserver with Master: aca3a4b0c4f8484c9a62e05683ce453e (127.18.168.131:45911)
I20260810 18:48:43.192638 26046 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.168.131@KRBTEST.COM'} at 127.18.168.131:46953
I20260810 18:48:43.193954 19106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.18.168.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.168.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.168.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.189.keytab.
Entry for principal oryx/127.18.168.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.189.keytab.
I20260810 18:48:43.223134 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.189:44715
--webserver_interface=127.18.168.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.189.keytab
--principal=oryx/127.18.168.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:37727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:43053,127.18.168.189:44715
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:43.337368 26569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:43.337637 26569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:43.337713 26569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:43.341814 26569 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:48:43.341889 26569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:43.341946 26569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:43.342002 26569 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:48:43.342037 26569 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:48:43.347105 26569 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:37727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:43053,127.18.168.189:44715
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.189.keytab
--principal=oryx/127.18.168.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.189:44715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.168.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
W20260810 18:48:43.347338 26569 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.18.168.190:43053,127.18.168.189:44715'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260810 18:48:43.348191 26569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:43.349174 26569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:43.357112 26575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:43.357106 26577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:43.357107 26579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:43.357288 26569 server_base.cc:1034] running on GCE node
I20260810 18:48:43.357846 26569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:43.358894 26569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:43.360054 26569 hybrid_clock.cc:648] HybridClock initialized: now 1786387723360026 us; error 39 us; skew 500 ppm
Aug 10 18:48:43 dist-test-slave-cbsf krb5kdc[25994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387723, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.168.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:43.362944 26569 init.cc:377] Logged in from keytab as oryx/127.18.168.189@KRBTEST.COM (short username oryx)
I20260810 18:48:43.363874 26569 webserver.cc:460] Webserver started at http://127.18.168.189:37803/ using document root <none> and password file <none>
I20260810 18:48:43.364421 26569 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:43.364481 26569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:43.364817 26569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:43.366791 26569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-1/data/instance:
uuid: "397a954133b4468d90dcbc43ac570b0f"
format_stamp: "Formatted at 2026-08-10 18:48:43 on dist-test-slave-cbsf"
server_key: "d6714ecf294996f577c00cec1e0815ea"
server_key_iv: "89c9eaa4f183ec58711f4d35a96e058a"
server_key_version: "encryptionkey@0"
I20260810 18:48:43.367358 26569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-1/wal/instance:
uuid: "397a954133b4468d90dcbc43ac570b0f"
format_stamp: "Formatted at 2026-08-10 18:48:43 on dist-test-slave-cbsf"
server_key: "d6714ecf294996f577c00cec1e0815ea"
server_key_iv: "89c9eaa4f183ec58711f4d35a96e058a"
server_key_version: "encryptionkey@0"
I20260810 18:48:43.371083 26569 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 18:48:43.373622 26587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:43.374782 26569 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:48:43.374912 26569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-1/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-1/wal
uuid: "397a954133b4468d90dcbc43ac570b0f"
format_stamp: "Formatted at 2026-08-10 18:48:43 on dist-test-slave-cbsf"
server_key: "d6714ecf294996f577c00cec1e0815ea"
server_key_iv: "89c9eaa4f183ec58711f4d35a96e058a"
server_key_version: "encryptionkey@0"
I20260810 18:48:43.375016 26569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:43.408550 26569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:43.411788 26569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:43.412104 26569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:43.419780 26569 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.189:44715
I20260810 18:48:43.420095 26644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.189:44715 every 8 connection(s)
I20260810 18:48:43.420941 26569 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-1/data/info.pb
I20260810 18:48:43.423763 26645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:43.427567 26645 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:48:43.430274 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 26569
Aug 10 18:48:43 dist-test-slave-cbsf krb5kdc[25994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387723, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.168.189@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM
Aug 10 18:48:43 dist-test-slave-cbsf krb5kdc[25994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387722, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.168.189@KRBTEST.COM
I20260810 18:48:43.442041 26088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:43.429388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52446 (local address 127.18.168.190:43053)
0810 18:48:43.429550 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:43.429553 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:43.430274 (+ 721us) server_negotiation.cc:407] Connection header received
0810 18:48:43.431587 (+ 1313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:43.431591 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:43.431661 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:43.431744 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:43.433035 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.433561 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.434768 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.435106 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.435183 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:43.437956 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:43.437960 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:43.437962 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:43.437991 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:43.439763 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:43.440289 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:43.440293 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:43.440294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:43.440383 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:43.440830 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:43.440834 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:43.440835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:43.441061 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:43.441179 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:43.441761 (+ 582us) server_negotiation.cc:299] Negotiation successful
0810 18:48:43.441888 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 18:48:43.442966 26649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:43.429693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43053 (local address 127.0.0.1:52446)
0810 18:48:43.430109 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:43.430148 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:43.431235 (+ 1087us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:43.431820 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:43.431828 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:43.432304 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:43.432884 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:43.432905 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.433686 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.433690 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:43.434655 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:43.434664 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.435228 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.435232 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:43.435465 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:43.436123 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:43.436137 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:43.437763 (+ 1626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:43.439871 (+ 2108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:43.439881 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:43.439890 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:43.440182 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:43.440478 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:43.440482 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:43.440484 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:43.440648 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:43.441178 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:43.441184 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:43.441464 (+ 280us) client_negotiation.cc:769] Sending connection context
0810 18:48:43.441673 (+ 209us) client_negotiation.cc:240] Negotiation successful
0810 18:48:43.441978 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":44,"spinlock_wait_cycles":2304,"thread_start_us":104,"threads_started":1}
I20260810 18:48:43.442965 26651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:43.431179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54004 (local address 127.18.168.189:44715)
0810 18:48:43.431547 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:43.431559 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:43.431596 (+ 37us) server_negotiation.cc:407] Connection header received
0810 18:48:43.431777 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:43.431782 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:43.432000 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:43.432191 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:43.432925 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.434180 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.435076 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.435577 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.435691 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:43.438388 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:43.438394 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:43.438407 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:43.438430 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:43.440404 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:43.441003 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:43.441009 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:43.441011 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:43.441097 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:43.441432 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:43.441435 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:43.441438 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:43.441709 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:43.441875 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:43.442551 (+ 676us) server_negotiation.cc:299] Negotiation successful
0810 18:48:43.442686 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":264,"spinlock_wait_cycles":1280,"thread_start_us":148,"threads_started":1}
I20260810 18:48:43.444588 26645 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.168.189" port: 44715 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:48:43 dist-test-slave-cbsf krb5kdc[25994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387723, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.168.189@KRBTEST.COM for oryx/127.18.168.189@KRBTEST.COM
I20260810 18:48:43.457319 26649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:43.445097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.189:44715 (local address 127.0.0.1:54016)
0810 18:48:43.448820 (+ 3723us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:43.448835 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:43.448917 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:43.449133 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:43.449136 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:43.449306 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:43.449422 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:43.449427 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.450286 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.450290 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:43.451113 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:43.451123 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.452245 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.452248 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:43.452327 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:43.452886 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:43.452898 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:43.454238 (+ 1340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:43.455886 (+ 1648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:43.455888 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:43.455890 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:43.456104 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:43.456392 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:43.456395 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:43.456396 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:43.456465 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:43.456901 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:43.456903 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:43.456976 (+ 73us) client_negotiation.cc:769] Sending connection context
0810 18:48:43.457077 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 18:48:43.457209 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3652}
I20260810 18:48:43.457515 26651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:43.445163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54016 (local address 127.18.168.189:44715)
0810 18:48:43.445650 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:43.445654 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:43.448899 (+ 3245us) server_negotiation.cc:407] Connection header received
0810 18:48:43.448991 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:43.448993 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:43.449036 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:43.449096 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:43.449585 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.450154 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.451247 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.452123 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.452233 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:43.454365 (+ 2132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:43.454368 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:43.454370 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:43.454393 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:43.455785 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:43.456198 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:43.456201 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:43.456202 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:43.456306 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:43.456548 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:43.456550 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:43.456552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:43.456811 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:43.456902 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:43.457261 (+ 359us) server_negotiation.cc:299] Negotiation successful
0810 18:48:43.457355 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424}
I20260810 18:48:43.460119 26645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f: Bootstrap starting.
I20260810 18:48:43.462564 26645 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:43.463333 26645 log.cc:824] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:43.465241 26645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f: No bootstrap required, opened a new log
I20260810 18:48:43.468196 26645 raft_consensus.cc:348] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 44715 } }
I20260810 18:48:43.468416 26645 raft_consensus.cc:374] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:43.468500 26645 raft_consensus.cc:729] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 397a954133b4468d90dcbc43ac570b0f, State: Initialized, Role: FOLLOWER
I20260810 18:48:43.469008 26645 consensus_queue.cc:260] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 44715 } }
I20260810 18:48:43.469451 26654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 44715 } } }
I20260810 18:48:43.469561 26654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:48:43.470180 26645 sys_catalog.cc:564] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [sys.catalog]: configured and running, proceeding with master startup.
W20260810 18:48:43.474764 26666 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:48:43.474923 26666 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:48:43.477344 26569 master_runner.cc:428] Detected that this master 397a954133b4468d90dcbc43ac570b0f is joining an existing cluster
I20260810 18:48:43.477411 26569 master_runner.cc:432] Initiating AddMaster RPC to add 127.18.168.189:44715
I20260810 18:48:43.488286 26651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:43.479830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54020 (local address 127.18.168.189:44715)
0810 18:48:43.479936 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:43.479942 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:43.480278 (+ 336us) server_negotiation.cc:407] Connection header received
0810 18:48:43.480382 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:43.480388 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:43.480437 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:43.480517 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:43.481095 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.481717 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.482693 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.482915 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.483034 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:43.484564 (+ 1530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:43.484567 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:43.484570 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:43.484599 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:43.486460 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:43.487007 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:43.487033 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:43.487037 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:43.487129 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:43.487435 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:43.487439 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:43.487441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:43.487668 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:43.487752 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:43.487987 (+ 235us) server_negotiation.cc:299] Negotiation successful
0810 18:48:43.488126 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":77,"server-negotiator.queue_time_us":34}
I20260810 18:48:43.488300 26672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:43.479897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.189:44715 (local address 127.0.0.1:54020)
0810 18:48:43.480178 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:43.480193 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:43.480297 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:43.480556 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:43.480559 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:43.480823 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:43.480963 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:43.480972 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.481827 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.481830 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:43.482578 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:43.482588 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.483036 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.483039 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:43.483123 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:43.484012 (+ 889us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:43.484024 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:43.484442 (+ 418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:43.486587 (+ 2145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:43.486592 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:43.486595 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:43.486891 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:43.487234 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:43.487237 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:43.487238 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:43.487331 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:43.487741 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:43.487744 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:43.487826 (+ 82us) client_negotiation.cc:769] Sending connection context
0810 18:48:43.487935 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 18:48:43.488076 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":128,"threads_started":1}
I20260810 18:48:43.489387 26674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:43.479947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43053 (local address 127.0.0.1:52460)
0810 18:48:43.480321 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:43.480334 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:43.480452 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:43.480864 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:43.480865 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:43.481024 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:43.481232 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:43.481239 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.482323 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.482327 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:43.483307 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:43.483318 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.483917 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.483921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:43.484039 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:43.484503 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:43.484518 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:43.485070 (+ 552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:43.487674 (+ 2604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:43.487677 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:43.487679 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:43.487932 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:43.488256 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:43.488259 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:43.488261 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:43.488344 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:43.488864 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:43.488866 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:43.488976 (+ 110us) client_negotiation.cc:769] Sending connection context
0810 18:48:43.489081 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 18:48:43.489212 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":40,"thread_start_us":77,"threads_started":1}
I20260810 18:48:43.489635 26088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:43.479836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52460 (local address 127.18.168.190:43053)
0810 18:48:43.479945 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:43.479950 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:43.480404 (+ 454us) server_negotiation.cc:407] Connection header received
0810 18:48:43.480582 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:43.480587 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:43.480632 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:43.480684 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:43.481356 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.482184 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.483469 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.483800 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.483882 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:43.485204 (+ 1322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:43.485208 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:43.485213 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:43.485242 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:43.487563 (+ 2321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:43.488051 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:43.488055 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:43.488057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:43.488145 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:43.488461 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:43.488465 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:43.488467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:43.488714 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:43.488791 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:43.489147 (+ 356us) server_negotiation.cc:299] Negotiation successful
0810 18:48:43.489269 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
Aug 10 18:48:43 dist-test-slave-cbsf krb5kdc[25994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387722, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.168.190@KRBTEST.COM for oryx/127.18.168.189@KRBTEST.COM
I20260810 18:48:43.504938 26675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:43.493815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.189:44715 (local address 127.0.0.1:54028)
0810 18:48:43.494534 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:43.494569 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:43.494674 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:43.494995 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:43.494999 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:43.495308 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:43.495404 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:43.495409 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.496636 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.496641 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:43.497662 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:43.497678 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.498424 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.498428 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:43.498549 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:43.499040 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:43.499055 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:43.500884 (+ 1829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:43.502916 (+ 2032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:43.502927 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:43.502934 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:43.503242 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:43.503558 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:43.503561 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:43.503563 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:43.503672 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:43.504164 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:43.504172 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:43.504304 (+ 132us) client_negotiation.cc:769] Sending connection context
0810 18:48:43.504464 (+ 160us) client_negotiation.cc:240] Negotiation successful
0810 18:48:43.504680 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":104,"threads_started":1}
I20260810 18:48:43.505285 26651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:43.493918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54028 (local address 127.18.168.189:44715)
0810 18:48:43.494042 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:43.494047 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:43.494644 (+ 597us) server_negotiation.cc:407] Connection header received
0810 18:48:43.494789 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:43.494792 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:43.494862 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:43.494959 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:43.495514 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.496265 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.497800 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.498178 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.498305 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:43.501027 (+ 2722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:43.501034 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:43.501039 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:43.501070 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:43.502786 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:43.503341 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:43.503349 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:43.503354 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:43.503450 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:43.503776 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:43.503781 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:43.503782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:43.504035 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:43.504150 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:43.504558 (+ 408us) server_negotiation.cc:299] Negotiation successful
0810 18:48:43.504746 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 18:48:43.507059 26046 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.18.168.189:44715: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "397a954133b4468d90dcbc43ac570b0f"
member_type: NON_VOTER
last_known_addr {
host: "127.18.168.189"
port: 44715
}
attrs {
promote: true
}
}
dest_uuid: "1dde94f763cc421f8e7b1693c060e3f3"
cas_config_opid_index: -1
I20260810 18:48:43.507848 26046 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: NON_VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: true } }
WARNING: no policy specified for oryx/127.18.168.188@KRBTEST.COM; defaulting to no policy
I20260810 18:48:43.510609 26618 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [term 0 FOLLOWER]: Advancing to term 1
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.168.188@KRBTEST.COM" created.
I20260810 18:48:43.510571 26678 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 397a954133b4468d90dcbc43ac570b0f (127.18.168.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: NON_VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: true } } }
I20260810 18:48:43.512075 26618 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [term 1 FOLLOWER]: Refusing update from remote peer 1dde94f763cc421f8e7b1693c060e3f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260810 18:48:43.513849 26678 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.18.168.189:44715
I20260810 18:48:43.514176 26678 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: NON_VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:43.514578 26569 master.cc:537] Master@127.18.168.189:44715 shutting down...
I20260810 18:48:43.515723 26678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1dde94f763cc421f8e7b1693c060e3f3" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: NON_VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: true } } }
I20260810 18:48:43.515862 26678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [sys.catalog]: This master's current role is: LEADER
W20260810 18:48:43.516471 26031 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260810 18:48:43.517823 26031 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 -> Peer 397a954133b4468d90dcbc43ac570b0f (127.18.168.189:44715): Couldn't send request to peer 397a954133b4468d90dcbc43ac570b0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260810 18:48:43.519562 26569 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:43.519881 26569 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:43.520030 26569 tablet_replica.cc:330] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f: stopping tablet replica
I20260810 18:48:43.535292 26569 master.cc:552] Master@127.18.168.189:44715 shutdown complete.
I20260810 18:48:43.535372 26569 master_runner.cc:305] Clearing existing system tablet
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.168.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.188.keytab.
Entry for principal oryx/127.18.168.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.188.keytab.
I20260810 18:48:43.535652 26569 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:43.535714 26569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:43.536863 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.188:46047
--webserver_interface=127.18.168.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.188.keytab
--principal=oryx/127.18.168.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:37727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:43053,127.18.168.189:44715,127.18.168.188:46047
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:48:43.537117 26569 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 18:48:43.538007 26686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:43.538250 26569 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:43.538400 26569 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 18:48:43.538439 26569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-1/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-1/wal
uuid: "397a954133b4468d90dcbc43ac570b0f"
format_stamp: "Formatted at 2026-08-10 18:48:43 on dist-test-slave-cbsf"
server_key: "d6714ecf294996f577c00cec1e0815ea"
server_key_iv: "89c9eaa4f183ec58711f4d35a96e058a"
server_key_version: "encryptionkey@0"
I20260810 18:48:43.540381 26569 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:43.542668 26569 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 18:48:43.542781 26569 log.cc:1196] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260810 18:48:43.543030 26569 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f: Deleting consensus metadata
I20260810 18:48:43.543198 26569 master_runner.cc:315] Copying system tablet from 127.18.168.190:43053
I20260810 18:48:43.544368 26569 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f: tablet copy: Beginning tablet copy session from remote peer at address 127.18.168.190:43053
I20260810 18:48:43.552765 26088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:43.545067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52468 (local address 127.18.168.190:43053)
0810 18:48:43.545208 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:43.545213 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:43.545440 (+ 227us) server_negotiation.cc:407] Connection header received
0810 18:48:43.545611 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:43.545614 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:43.545671 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:43.545762 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:43.546216 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.546757 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.547602 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.547839 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.547957 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:43.549147 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:43.549150 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:43.549153 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:43.549185 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:43.550923 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:43.551421 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:43.551425 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:43.551426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:43.551539 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:43.551839 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:43.551843 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:43.551853 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:43.552045 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:43.552153 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:43.552444 (+ 291us) server_negotiation.cc:299] Negotiation successful
0810 18:48:43.552563 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 18:48:43.552778 26693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:43.544981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43053 (local address 127.0.0.1:52468)
0810 18:48:43.545335 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:43.545348 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:43.545488 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:43.545780 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:43.545783 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:43.545987 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:43.546091 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:43.546097 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.546871 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.546876 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:43.547462 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:43.547470 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.547962 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.547965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:43.548064 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:43.548609 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:43.548621 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:43.549003 (+ 382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:43.551081 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:43.551085 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:43.551088 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:43.551316 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:43.551636 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:43.551640 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:43.551642 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:43.551728 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:43.552143 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:43.552146 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:43.552256 (+ 110us) client_negotiation.cc:769] Sending connection context
0810 18:48:43.552386 (+ 130us) client_negotiation.cc:240] Negotiation successful
0810 18:48:43.552530 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":96,"threads_started":1}
I20260810 18:48:43.553594 26068 tablet_copy_service.cc:140] P 1dde94f763cc421f8e7b1693c060e3f3: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 397a954133b4468d90dcbc43ac570b0f ({username='oryx', principal='oryx/127.18.168.189@KRBTEST.COM'} at 127.0.0.1:52468)
I20260810 18:48:43.553789 26068 tablet_copy_service.cc:161] P 1dde94f763cc421f8e7b1693c060e3f3: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 397a954133b4468d90dcbc43ac570b0f at {username='oryx', principal='oryx/127.18.168.189@KRBTEST.COM'} at 127.0.0.1:52468: session id = 397a954133b4468d90dcbc43ac570b0f-00000000000000000000000000000000
I20260810 18:48:43.555225 26068 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 18:48:43.556664 26569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 18:48:43.558454 26569 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f: tablet copy: Starting download of 0 data blocks...
I20260810 18:48:43.558672 26569 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f: tablet copy: Starting download of 1 WAL segments...
I20260810 18:48:43.560238 26569 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 18:48:43.561820 26068 tablet_copy_service.cc:342] P 1dde94f763cc421f8e7b1693c060e3f3: Request end of tablet copy session 397a954133b4468d90dcbc43ac570b0f-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.18.168.189@KRBTEST.COM'} at 127.0.0.1:52468
I20260810 18:48:43.561939 26068 tablet_copy_service.cc:433] P 1dde94f763cc421f8e7b1693c060e3f3: ending tablet copy session 397a954133b4468d90dcbc43ac570b0f-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 397a954133b4468d90dcbc43ac570b0f
I20260810 18:48:43.568882 26569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:43.570568 26694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:43.571096 26697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:43.572856 26695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:43.576840 26569 server_base.cc:1034] running on GCE node
I20260810 18:48:43.577014 26569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:43.577417 26569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:43.578554 26569 hybrid_clock.cc:648] HybridClock initialized: now 1786387723578582 us; error 78 us; skew 500 ppm
Aug 10 18:48:43 dist-test-slave-cbsf krb5kdc[25994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387723, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.168.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:43.580071 26569 init.cc:377] Logged in from keytab as oryx/127.18.168.189@KRBTEST.COM (short username oryx)
I20260810 18:48:43.580543 26569 webserver.cc:460] Webserver started at http://127.18.168.189:40839/ using document root <none> and password file <none>
I20260810 18:48:43.580757 26569 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:43.580807 26569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:43.581941 26569 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 18:48:43.582952 26706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:43.583197 26569 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 18:48:43.583284 26569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-1/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-1/wal
uuid: "397a954133b4468d90dcbc43ac570b0f"
format_stamp: "Formatted at 2026-08-10 18:48:43 on dist-test-slave-cbsf"
server_key: "d6714ecf294996f577c00cec1e0815ea"
server_key_iv: "89c9eaa4f183ec58711f4d35a96e058a"
server_key_version: "encryptionkey@0"
I20260810 18:48:43.583365 26569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:43.589179 26569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:43.591034 26569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:43.596305 26569 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.189:44715
I20260810 18:48:43.596563 26767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.189:44715 every 8 connection(s)
I20260810 18:48:43.596949 26569 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-1/data/info.pb
I20260810 18:48:43.598524 26769 sys_catalog.cc:263] Verifying existing consensus state
I20260810 18:48:43.599377 26769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f: Bootstrap starting.
I20260810 18:48:43.607582 26769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:48:43.607858 26769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f: Bootstrap complete.
I20260810 18:48:43.608317 26769 raft_consensus.cc:348] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: NON_VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: true } }
I20260810 18:48:43.608448 26769 raft_consensus.cc:729] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 397a954133b4468d90dcbc43ac570b0f, State: Initialized, Role: LEARNER
I20260810 18:48:43.608577 26769 consensus_queue.cc:260] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: NON_VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: true } }
I20260810 18:48:43.608881 26772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: NON_VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: true } } }
I20260810 18:48:43.609043 26769 sys_catalog.cc:564] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:48:43.609047 26772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [sys.catalog]: This master's current role is: LEARNER
I20260810 18:48:43.612416 26784 catalog_manager.cc:1259] Loaded cluster ID: 4f874a85450b418499bc2b5946eac0c8
I20260810 18:48:43.612486 26784 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f: loading cluster ID for follower catalog manager: success
I20260810 18:48:43.614081 26784 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f: acquiring CA information for follower catalog manager: success
I20260810 18:48:43.615093 26784 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 18:48:43.714674 26687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:43.714901 26687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:43.714968 26687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:43.719262 26687 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:48:43.719324 26687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:43.719344 26687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:43.719360 26687 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:48:43.719416 26687 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:48:43.724941 26687 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:37727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:43053,127.18.168.189:44715,127.18.168.188:46047
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.188.keytab
--principal=oryx/127.18.168.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.188:46047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.168.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:43.725966 26687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:43.726975 26687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:43.735929 26789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:43.735893 26792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:43.736002 26790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:43.736320 26687 server_base.cc:1034] running on GCE node
I20260810 18:48:43.736935 26687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:43.737957 26687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:43.739147 26687 hybrid_clock.cc:648] HybridClock initialized: now 1786387723739121 us; error 39 us; skew 500 ppm
I20260810 18:48:43.739530 26260 heartbeater.cc:503] Master 127.18.168.190:43053 was elected leader, sending a full tablet report...
Aug 10 18:48:43 dist-test-slave-cbsf krb5kdc[25994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387723, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.168.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:43.742842 26687 init.cc:377] Logged in from keytab as oryx/127.18.168.188@KRBTEST.COM (short username oryx)
I20260810 18:48:43.743700 26687 webserver.cc:460] Webserver started at http://127.18.168.188:34391/ using document root <none> and password file <none>
I20260810 18:48:43.744274 26687 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:43.744376 26687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:43.744628 26687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:43.746757 26687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-2/data/instance:
uuid: "c8aece922db748f48528a46f14ec0d88"
format_stamp: "Formatted at 2026-08-10 18:48:43 on dist-test-slave-cbsf"
server_key: "c196f86e7c92732ed13f7ae3f2f96dda"
server_key_iv: "22aab3bca0d0304b473cea66b79b11d0"
server_key_version: "encryptionkey@0"
I20260810 18:48:43.747375 26687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-2/wal/instance:
uuid: "c8aece922db748f48528a46f14ec0d88"
format_stamp: "Formatted at 2026-08-10 18:48:43 on dist-test-slave-cbsf"
server_key: "c196f86e7c92732ed13f7ae3f2f96dda"
server_key_iv: "22aab3bca0d0304b473cea66b79b11d0"
server_key_version: "encryptionkey@0"
I20260810 18:48:43.751142 26687 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260810 18:48:43.753489 26802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:43.754742 26687 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 18:48:43.754882 26687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-2/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-2/wal
uuid: "c8aece922db748f48528a46f14ec0d88"
format_stamp: "Formatted at 2026-08-10 18:48:43 on dist-test-slave-cbsf"
server_key: "c196f86e7c92732ed13f7ae3f2f96dda"
server_key_iv: "22aab3bca0d0304b473cea66b79b11d0"
server_key_version: "encryptionkey@0"
I20260810 18:48:43.754994 26687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:43.772622 26687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:43.775822 26687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:43.776067 26687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:43.790660 26687 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.188:46047
I20260810 18:48:43.790910 26858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.188:46047 every 8 connection(s)
I20260810 18:48:43.791625 26687 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-2/data/info.pb
I20260810 18:48:43.794447 26859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:43.795627 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 26687
I20260810 18:48:43.798992 26859 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:48:43 dist-test-slave-cbsf krb5kdc[25994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387722, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.168.188@KRBTEST.COM
Aug 10 18:48:43 dist-test-slave-cbsf krb5kdc[25994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387723, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.168.188@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM
I20260810 18:48:43.811654 26088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:43.800898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52476 (local address 127.18.168.190:43053)
0810 18:48:43.801014 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:43.801019 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:43.801159 (+ 140us) server_negotiation.cc:407] Connection header received
0810 18:48:43.801336 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:43.801339 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:43.801393 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:43.801479 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:43.802315 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.802834 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.804037 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.804256 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.804361 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:43.807812 (+ 3451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:43.807815 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:43.807816 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:43.807838 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:43.809294 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:43.810005 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:43.810009 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:43.810010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:43.810090 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:43.810495 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:43.810499 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:43.810501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:43.810698 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:43.810828 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:43.811352 (+ 524us) server_negotiation.cc:299] Negotiation successful
0810 18:48:43.811470 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 18:48:43.812410 26864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:43.800559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43053 (local address 127.0.0.1:52476)
0810 18:48:43.800956 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:43.801006 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:43.801225 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:43.801537 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:43.801542 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:43.801870 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:43.802171 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:43.802183 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.803054 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.803057 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:43.803893 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:43.803904 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.804411 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.804415 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:43.804563 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:43.805231 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:43.805247 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:43.807632 (+ 2385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:43.809414 (+ 1782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:43.809423 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:43.809433 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:43.809841 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:43.810185 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:43.810189 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:43.810191 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:43.810393 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:43.810826 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:43.810833 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:43.811124 (+ 291us) client_negotiation.cc:769] Sending connection context
0810 18:48:43.811306 (+ 182us) client_negotiation.cc:240] Negotiation successful
0810 18:48:43.811598 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":2,"thread_start_us":120,"threads_started":1}
I20260810 18:48:43.812804 26862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:43.797269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47806 (local address 127.18.168.188:46047)
0810 18:48:43.797670 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:43.797694 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:43.797757 (+ 63us) server_negotiation.cc:407] Connection header received
0810 18:48:43.798424 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:43.798440 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:43.798750 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:43.799203 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:43.800031 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.802395 (+ 2364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.803257 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.803857 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.804034 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:43.807245 (+ 3211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:43.807252 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:43.807266 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:43.807294 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:43.809809 (+ 2515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:43.810309 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:43.810317 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:43.810319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:43.810401 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:43.810693 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:43.810698 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:43.810700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:43.811951 (+ 1251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:43.812161 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:43.812350 (+ 189us) server_negotiation.cc:299] Negotiation successful
0810 18:48:43.812488 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":264,"thread_start_us":140,"threads_started":1}
I20260810 18:48:43.814278 26859 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.168.189" port: 44715 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:48:43 dist-test-slave-cbsf krb5kdc[25994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387723, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.168.188@KRBTEST.COM for oryx/127.18.168.189@KRBTEST.COM
I20260810 18:48:43.822443 26864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:43.814781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.189:44715 (local address 127.0.0.1:54044)
0810 18:48:43.814936 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:43.814949 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:43.815021 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:43.815536 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:43.815538 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:43.815703 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:43.815781 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:43.815786 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.816696 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.816701 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:43.817265 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:43.817274 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.817745 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.817749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:43.817896 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:43.818266 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:43.818277 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:43.819473 (+ 1196us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:43.821289 (+ 1816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:43.821291 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:43.821292 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:43.821468 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:43.821718 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:43.821720 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:43.821721 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:43.821789 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:43.822104 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:43.822106 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:43.822175 (+ 69us) client_negotiation.cc:769] Sending connection context
0810 18:48:43.822252 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 18:48:43.822341 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91}
I20260810 18:48:43.822516 26865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:43.814902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54044 (local address 127.18.168.189:44715)
0810 18:48:43.815231 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:43.815235 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:43.815246 (+ 11us) server_negotiation.cc:407] Connection header received
0810 18:48:43.815282 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:43.815284 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:43.815322 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:43.815530 (+ 208us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:43.815909 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.816560 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.817397 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.817633 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.817755 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:43.819616 (+ 1861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:43.819619 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:43.819621 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:43.819643 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:43.821194 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:43.821555 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:43.821558 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:43.821559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:43.821631 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:43.821864 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:43.821866 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:43.821868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:43.822026 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:43.822100 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:43.822310 (+ 210us) server_negotiation.cc:299] Negotiation successful
0810 18:48:43.822402 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":165,"threads_started":1}
I20260810 18:48:43.823483 26859 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46047 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:48:43 dist-test-slave-cbsf krb5kdc[25994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387723, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.168.188@KRBTEST.COM for oryx/127.18.168.188@KRBTEST.COM
I20260810 18:48:43.830966 26864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:43.823851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:46047 (local address 127.0.0.1:47814)
0810 18:48:43.823946 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:43.823959 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:43.824044 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:43.824269 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:43.824271 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:43.824384 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:43.824463 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:43.824467 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.825709 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.825711 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:43.826220 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:43.826227 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.826700 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.826702 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:43.826756 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:43.827095 (+ 339us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:43.827105 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:43.828111 (+ 1006us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:43.829801 (+ 1690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:43.829804 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:43.829805 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:43.829987 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:43.830218 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:43.830220 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:43.830221 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:43.830292 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:43.830594 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:43.830596 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:43.830661 (+ 65us) client_negotiation.cc:769] Sending connection context
0810 18:48:43.830737 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 18:48:43.830823 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 18:48:43.830983 26862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:43.823985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47814 (local address 127.18.168.188:46047)
0810 18:48:43.824099 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:43.824102 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:43.824112 (+ 10us) server_negotiation.cc:407] Connection header received
0810 18:48:43.824138 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:43.824140 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:43.824181 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:43.824241 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:43.824554 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.825586 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.826316 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.826607 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.826688 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:43.828291 (+ 1603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:43.828293 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:43.828295 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:43.828316 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:43.829708 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:43.830069 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:43.830071 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:43.830072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:43.830139 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:43.830366 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:43.830368 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:43.830369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:43.830515 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:43.830590 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:43.830794 (+ 204us) server_negotiation.cc:299] Negotiation successful
0810 18:48:43.830875 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 18:48:43.834435 26859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88: Bootstrap starting.
I20260810 18:48:43.837095 26859 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:43.838212 26859 log.cc:824] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:43.840248 26859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88: No bootstrap required, opened a new log
I20260810 18:48:43.844816 26859 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 44715 } } peers { permanent_uuid: "c8aece922db748f48528a46f14ec0d88" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46047 } }
I20260810 18:48:43.845008 26859 raft_consensus.cc:374] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:43.845083 26859 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8aece922db748f48528a46f14ec0d88, State: Initialized, Role: FOLLOWER
I20260810 18:48:43.845492 26859 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 44715 } } peers { permanent_uuid: "c8aece922db748f48528a46f14ec0d88" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46047 } }
I20260810 18:48:43.846109 26868 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 44715 } } peers { permanent_uuid: "c8aece922db748f48528a46f14ec0d88" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46047 } } }
I20260810 18:48:43.846285 26868 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:48:43.846736 26859 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 18:48:43.853430 26880 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:48:43.853518 26880 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:48:43.857452 26687 master_runner.cc:428] Detected that this master c8aece922db748f48528a46f14ec0d88 is joining an existing cluster
I20260810 18:48:43.857513 26687 master_runner.cc:432] Initiating AddMaster RPC to add 127.18.168.188:46047
I20260810 18:48:43.866321 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 26115
I20260810 18:48:43.878002 26865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:43.861691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54050 (local address 127.18.168.189:44715)
0810 18:48:43.861817 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:43.861821 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:43.862325 (+ 504us) server_negotiation.cc:407] Connection header received
0810 18:48:43.862671 (+ 346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:43.862676 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:43.862757 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:43.862869 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:43.863987 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.864971 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.868396 (+ 3425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.872348 (+ 3952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.872445 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:43.874045 (+ 1600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:43.874050 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:43.874056 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:43.874084 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:43.875984 (+ 1900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:43.876512 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:43.876515 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:43.876517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:43.876605 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:43.876982 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:43.876986 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:43.876987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:43.877229 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:43.877308 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:43.877694 (+ 386us) server_negotiation.cc:299] Negotiation successful
0810 18:48:43.877835 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 18:48:43.878541 26887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:43.861009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.189:44715 (local address 127.0.0.1:54050)
0810 18:48:43.862105 (+ 1096us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:43.862122 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:43.862525 (+ 403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:43.863022 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:43.863025 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:43.863408 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:43.863828 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:43.863838 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.865139 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.865142 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:43.868264 (+ 3122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:43.868275 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.872601 (+ 4326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.872604 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:43.872730 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:43.873273 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:43.873293 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:43.873898 (+ 605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:43.876110 (+ 2212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:43.876114 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:43.876116 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:43.876394 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:43.876774 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:43.876777 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:43.876779 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:43.876873 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:43.877382 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:43.877385 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:43.877484 (+ 99us) client_negotiation.cc:769] Sending connection context
0810 18:48:43.878163 (+ 679us) client_negotiation.cc:240] Negotiation successful
0810 18:48:43.878320 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":963,"mutex_wait_us":28,"thread_start_us":335,"threads_started":1}
I20260810 18:48:43.892583 26088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:43.861020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52492 (local address 127.18.168.190:43053)
0810 18:48:43.861144 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:43.861148 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:43.862890 (+ 1742us) server_negotiation.cc:407] Connection header received
0810 18:48:43.863071 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:43.863075 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:43.863146 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:43.863215 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:43.864290 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.864897 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.872163 (+ 7266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.872507 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.872604 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:43.887428 (+ 14824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:43.887436 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:43.887442 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:43.887495 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:43.889704 (+ 2209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:43.890863 (+ 1159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:43.890869 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:43.890874 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:43.891000 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:43.891374 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:43.891381 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:43.891385 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:43.891672 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:43.891761 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:43.892259 (+ 498us) server_negotiation.cc:299] Negotiation successful
0810 18:48:43.892402 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 18:48:43.892773 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.129:42891
--local_ip_for_outbound_sockets=127.18.168.129
--tserver_master_addrs=127.18.168.190:43053
--webserver_port=44363
--webserver_interface=127.18.168.129
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.129.keytab
--principal=oryx/127.18.168.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:37727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.168.190:43053,127.18.168.189:44715
--tserver_master_addrs=127.18.168.190:43053,127.18.168.189:44715,127.18.168.188:46047 with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:48:43.892876 26862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:43.861451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47830 (local address 127.18.168.188:46047)
0810 18:48:43.861601 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:43.861607 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:43.862783 (+ 1176us) server_negotiation.cc:407] Connection header received
0810 18:48:43.862940 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:43.862943 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:43.862977 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:43.863052 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:43.885410 (+ 22358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.886352 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.887756 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.888078 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.888153 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:43.889560 (+ 1407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:43.889564 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:43.889566 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:43.889586 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:43.891266 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:43.891706 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:43.891709 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:43.891711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:43.891776 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:43.892014 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:43.892017 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:43.892018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:43.892191 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:43.892251 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:43.892530 (+ 279us) server_negotiation.cc:299] Negotiation successful
0810 18:48:43.892637 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 18:48:43.893352 26889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:43.861106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:46047 (local address 127.0.0.1:47830)
0810 18:48:43.862527 (+ 1421us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:43.862542 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:43.862844 (+ 302us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:43.863298 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:43.863301 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:43.885138 (+ 21837us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:43.885288 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:43.885297 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.886530 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.886534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:43.887624 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:43.887635 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.888373 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.888377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:43.888523 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:43.888980 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:43.888995 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:43.889417 (+ 422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:43.891380 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:43.891383 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:43.891385 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:43.891615 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:43.891863 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:43.891866 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:43.891868 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:43.891935 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:43.892303 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:43.892305 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:43.892373 (+ 68us) client_negotiation.cc:769] Sending connection context
0810 18:48:43.893064 (+ 691us) client_negotiation.cc:240] Negotiation successful
0810 18:48:43.893189 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1343,"thread_start_us":423,"threads_started":1}
I20260810 18:48:43.896556 26888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:43.861973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43053 (local address 127.0.0.1:52492)
0810 18:48:43.862671 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:43.862685 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:43.862957 (+ 272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:43.863407 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:43.863410 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:43.863818 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:43.864171 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:43.864180 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.865005 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.865011 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:43.866418 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:43.872026 (+ 5608us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.872755 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.872758 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:43.872855 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:43.886573 (+ 13718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:43.886593 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:43.887238 (+ 645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:43.890093 (+ 2855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:43.890098 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:43.890103 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:43.890745 (+ 642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:43.891132 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:43.891137 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:43.891142 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:43.891249 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:43.891845 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:43.891850 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:43.891955 (+ 105us) client_negotiation.cc:769] Sending connection context
0810 18:48:43.895963 (+ 4008us) client_negotiation.cc:240] Negotiation successful
0810 18:48:43.896246 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"mutex_wait_us":160,"thread_start_us":193,"threads_started":1}
Aug 10 18:48:43 dist-test-slave-cbsf krb5kdc[25994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387722, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.168.190@KRBTEST.COM for oryx/127.18.168.188@KRBTEST.COM
I20260810 18:48:43.910648 26675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:43.901197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:46047 (local address 127.0.0.1:47840)
0810 18:48:43.901333 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:43.901347 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:43.901429 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:43.901801 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:43.901804 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:43.902023 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:43.902133 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:43.902140 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.903140 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.903144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:43.904054 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:43.904065 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.904754 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.904758 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:43.904850 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:43.905354 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:43.905370 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:43.907013 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:43.909051 (+ 2038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:43.909054 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:43.909057 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:43.909295 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:43.909597 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:43.909600 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:43.909602 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:43.909687 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:43.910117 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:43.910121 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:43.910218 (+ 97us) client_negotiation.cc:769] Sending connection context
0810 18:48:43.910316 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 18:48:43.910454 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260810 18:48:43.911196 26862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:43.900924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47840 (local address 127.18.168.188:46047)
0810 18:48:43.901034 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:43.901038 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:43.901517 (+ 479us) server_negotiation.cc:407] Connection header received
0810 18:48:43.901584 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:43.901587 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:43.901635 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:43.901697 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:43.902259 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.903006 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.904188 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.904517 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.904605 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:43.907161 (+ 2556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:43.907165 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:43.907168 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:43.907191 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:43.908927 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:43.909403 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:43.909407 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:43.909409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:43.909498 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:43.909782 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:43.909786 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:43.909787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:43.909980 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:43.910071 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:43.910950 (+ 879us) server_negotiation.cc:299] Negotiation successful
0810 18:48:43.911057 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 18:48:43.912199 26046 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.18.168.188:46047: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "c8aece922db748f48528a46f14ec0d88"
member_type: NON_VOTER
last_known_addr {
host: "127.18.168.188"
port: 46047
}
attrs {
promote: true
}
}
dest_uuid: "1dde94f763cc421f8e7b1693c060e3f3"
cas_config_opid_index: 5
I20260810 18:48:43.912963 26046 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: NON_VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: true } } peers { permanent_uuid: "c8aece922db748f48528a46f14ec0d88" member_type: NON_VOTER last_known_addr { host: "127.18.168.188" port: 46047 } attrs { promote: true } }
I20260810 18:48:43.914119 26681 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER c8aece922db748f48528a46f14ec0d88 (127.18.168.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: NON_VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: true } } peers { permanent_uuid: "c8aece922db748f48528a46f14ec0d88" member_type: NON_VOTER last_known_addr { host: "127.18.168.188" port: 46047 } attrs { promote: true } } }
I20260810 18:48:43.915714 26681 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.18.168.188:46047
I20260810 18:48:43.915889 26681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1dde94f763cc421f8e7b1693c060e3f3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: NON_VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: true } } peers { permanent_uuid: "c8aece922db748f48528a46f14ec0d88" member_type: NON_VOTER last_known_addr { host: "127.18.168.188" port: 46047 } attrs { promote: true } } }
I20260810 18:48:43.915992 26681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:43.916929 26687 master.cc:537] Master@127.18.168.188:46047 shutting down...
I20260810 18:48:43.917412 26830 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:43.918720 26830 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [term 1 FOLLOWER]: Refusing update from remote peer 1dde94f763cc421f8e7b1693c060e3f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 18:48:43.919402 26679 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8aece922db748f48528a46f14ec0d88" member_type: NON_VOTER last_known_addr { host: "127.18.168.188" port: 46047 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 18:48:43.921283 26029 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 -> Peer c8aece922db748f48528a46f14ec0d88 (127.18.168.188:46047): Couldn't send request to peer c8aece922db748f48528a46f14ec0d88. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260810 18:48:43.922189 26865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:43.916465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54066 (local address 127.18.168.189:44715)
0810 18:48:43.916642 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:43.916647 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:43.917602 (+ 955us) server_negotiation.cc:407] Connection header received
0810 18:48:43.917780 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:43.917783 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:43.917857 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:43.917925 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:43.918295 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.919086 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.920653 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.921354 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.921445 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:43.921606 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:43.921977 (+ 371us) server_negotiation.cc:299] Negotiation successful
0810 18:48:43.922065 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260810 18:48:43.922580 26675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:43.916288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.189:44715 (local address 127.0.0.1:54066)
0810 18:48:43.917529 (+ 1241us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:43.917544 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:43.917666 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:43.918082 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:43.918085 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:43.918101 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:43.918180 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:43.918185 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.919229 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.919232 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:43.920510 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:43.920522 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.921705 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.921708 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:43.921799 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:43.921860 (+ 61us) client_negotiation.cc:769] Sending connection context
0810 18:48:43.922376 (+ 516us) client_negotiation.cc:240] Negotiation successful
0810 18:48:43.922454 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1151}
I20260810 18:48:43.923072 26737 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [term 1 LEARNER]: Refusing update from remote peer 1dde94f763cc421f8e7b1693c060e3f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 18:48:43.923687 26679 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: NON_VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260810 18:48:43.925695 26737 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER c8aece922db748f48528a46f14ec0d88 (127.18.168.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: NON_VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: true } } peers { permanent_uuid: "c8aece922db748f48528a46f14ec0d88" member_type: NON_VOTER last_known_addr { host: "127.18.168.188" port: 46047 } attrs { promote: true } } }
I20260810 18:48:43.928085 26772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1dde94f763cc421f8e7b1693c060e3f3. Latest consensus state: current_term: 1 leader_uuid: "1dde94f763cc421f8e7b1693c060e3f3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: NON_VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: true } } peers { permanent_uuid: "c8aece922db748f48528a46f14ec0d88" member_type: NON_VOTER last_known_addr { host: "127.18.168.188" port: 46047 } attrs { promote: true } } }
I20260810 18:48:43.928220 26772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [sys.catalog]: This master's current role is: LEARNER
I20260810 18:48:43.928314 26772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1dde94f763cc421f8e7b1693c060e3f3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: NON_VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: true } } peers { permanent_uuid: "c8aece922db748f48528a46f14ec0d88" member_type: NON_VOTER last_known_addr { host: "127.18.168.188" port: 46047 } attrs { promote: true } } }
I20260810 18:48:43.928391 26772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [sys.catalog]: This master's current role is: LEARNER
I20260810 18:48:43.928505 26687 raft_consensus.cc:2229] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:43.928897 26687 raft_consensus.cc:2258] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:43.929019 26687 tablet_replica.cc:330] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88: stopping tablet replica
I20260810 18:48:43.930213 26679 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3: attempting to promote NON_VOTER 397a954133b4468d90dcbc43ac570b0f to VOTER
I20260810 18:48:43.930678 26679 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: false } } peers { permanent_uuid: "c8aece922db748f48528a46f14ec0d88" member_type: NON_VOTER last_known_addr { host: "127.18.168.188" port: 46047 } attrs { promote: true } }
I20260810 18:48:43.932257 26737 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [term 1 LEARNER]: Refusing update from remote peer 1dde94f763cc421f8e7b1693c060e3f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260810 18:48:43.932710 26678 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260810 18:48:43.934675 26675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:43.934349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:46047 (local address 127.0.0.1:47842)
0810 18:48:43.934516 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:43.934595 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.188:46047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260810 18:48:43.934806 26029 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 -> Peer c8aece922db748f48528a46f14ec0d88 (127.18.168.188:46047): Couldn't send request to peer c8aece922db748f48528a46f14ec0d88. Status: Network error: Client connection negotiation failed: client connection to 127.18.168.188:46047: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 18:48:43.934753 26681 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 397a954133b4468d90dcbc43ac570b0f (127.18.168.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: false } } peers { permanent_uuid: "c8aece922db748f48528a46f14ec0d88" member_type: NON_VOTER last_known_addr { host: "127.18.168.188" port: 46047 } attrs { promote: true } } }
I20260810 18:48:43.936506 26687 master.cc:552] Master@127.18.168.188:46047 shutdown complete.
I20260810 18:48:43.936559 26687 master_runner.cc:305] Clearing existing system tablet
I20260810 18:48:43.936721 26737 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 397a954133b4468d90dcbc43ac570b0f (127.18.168.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: false } } peers { permanent_uuid: "c8aece922db748f48528a46f14ec0d88" member_type: NON_VOTER last_known_addr { host: "127.18.168.188" port: 46047 } attrs { promote: true } } }
I20260810 18:48:43.936873 26687 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:43.936929 26687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:43.938423 26687 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 18:48:43.938489 26772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1dde94f763cc421f8e7b1693c060e3f3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: false } } peers { permanent_uuid: "c8aece922db748f48528a46f14ec0d88" member_type: NON_VOTER last_known_addr { host: "127.18.168.188" port: 46047 } attrs { promote: true } } }
I20260810 18:48:43.938588 26772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:48:43.938990 26678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1dde94f763cc421f8e7b1693c060e3f3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: false } } peers { permanent_uuid: "c8aece922db748f48528a46f14ec0d88" member_type: NON_VOTER last_known_addr { host: "127.18.168.188" port: 46047 } attrs { promote: true } } }
I20260810 18:48:43.939091 26678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:43.939378 26681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1dde94f763cc421f8e7b1693c060e3f3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: false } } peers { permanent_uuid: "c8aece922db748f48528a46f14ec0d88" member_type: NON_VOTER last_known_addr { host: "127.18.168.188" port: 46047 } attrs { promote: true } } }
I20260810 18:48:43.939540 26681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:43.939994 26896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:43.940248 26687 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:43.940383 26687 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 18:48:43.940428 26687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-2/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-2/wal
uuid: "c8aece922db748f48528a46f14ec0d88"
format_stamp: "Formatted at 2026-08-10 18:48:43 on dist-test-slave-cbsf"
server_key: "c196f86e7c92732ed13f7ae3f2f96dda"
server_key_iv: "22aab3bca0d0304b473cea66b79b11d0"
server_key_version: "encryptionkey@0"
I20260810 18:48:43.942646 26687 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:43.945139 26687 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 18:48:43.945235 26687 log.cc:1196] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260810 18:48:43.945498 26687 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88: Deleting consensus metadata
I20260810 18:48:43.945693 26687 master_runner.cc:315] Copying system tablet from 127.18.168.190:43053
I20260810 18:48:43.947055 26687 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88: tablet copy: Beginning tablet copy session from remote peer at address 127.18.168.190:43053
I20260810 18:48:43.957890 26902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:43.947649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43053 (local address 127.0.0.1:52508)
0810 18:48:43.948862 (+ 1213us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:43.948877 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:43.948989 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:43.949308 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:43.949311 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:43.949556 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:43.949681 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:43.949688 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.950861 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.950865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:43.951771 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:43.951781 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.952445 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.952449 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:43.952558 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:43.953270 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:43.953284 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:43.953738 (+ 454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:43.955999 (+ 2261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:43.956003 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:43.956005 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:43.956254 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:43.956664 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:43.956668 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:43.956670 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:43.956796 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:43.957274 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:43.957278 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:43.957379 (+ 101us) client_negotiation.cc:769] Sending connection context
0810 18:48:43.957491 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 18:48:43.957632 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1138,"thread_start_us":86,"threads_started":1}
I20260810 18:48:43.957953 26088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:43.947738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52508 (local address 127.18.168.190:43053)
0810 18:48:43.947864 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:43.947869 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:43.948968 (+ 1099us) server_negotiation.cc:407] Connection header received
0810 18:48:43.949081 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:43.949087 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:43.949170 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:43.949270 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:43.949820 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.950731 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.951903 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.952307 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.952428 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:43.953893 (+ 1465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:43.953897 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:43.953901 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:43.953935 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:43.955843 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:43.956362 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:43.956366 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:43.956368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:43.956556 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:43.956898 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:43.956902 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:43.956904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:43.957130 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:43.957279 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:43.957663 (+ 384us) server_negotiation.cc:299] Negotiation successful
0810 18:48:43.957792 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 18:48:43.958774 26413 heartbeater.cc:503] Master 127.18.168.190:43053 was elected leader, sending a full tablet report...
I20260810 18:48:43.958770 26068 tablet_copy_service.cc:140] P 1dde94f763cc421f8e7b1693c060e3f3: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer c8aece922db748f48528a46f14ec0d88 ({username='oryx', principal='oryx/127.18.168.188@KRBTEST.COM'} at 127.0.0.1:52508)
I20260810 18:48:43.958864 26068 tablet_copy_service.cc:161] P 1dde94f763cc421f8e7b1693c060e3f3: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer c8aece922db748f48528a46f14ec0d88 at {username='oryx', principal='oryx/127.18.168.188@KRBTEST.COM'} at 127.0.0.1:52508: session id = c8aece922db748f48528a46f14ec0d88-00000000000000000000000000000000
I20260810 18:48:43.959960 26068 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 18:48:43.961398 26687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 18:48:43.964154 26687 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88: tablet copy: Starting download of 0 data blocks...
I20260810 18:48:43.964365 26687 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88: tablet copy: Starting download of 1 WAL segments...
I20260810 18:48:43.966074 26687 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 18:48:43.967983 26068 tablet_copy_service.cc:342] P 1dde94f763cc421f8e7b1693c060e3f3: Request end of tablet copy session c8aece922db748f48528a46f14ec0d88-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.18.168.188@KRBTEST.COM'} at 127.0.0.1:52508
I20260810 18:48:43.968091 26068 tablet_copy_service.cc:433] P 1dde94f763cc421f8e7b1693c060e3f3: ending tablet copy session c8aece922db748f48528a46f14ec0d88-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer c8aece922db748f48528a46f14ec0d88
I20260810 18:48:43.975898 26687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:43.977666 26903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:43.977838 26687 server_base.cc:1034] running on GCE node
W20260810 18:48:43.981508 26904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:43.981536 26906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:43.981884 26687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:43.982189 26687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:43.983309 26687 hybrid_clock.cc:648] HybridClock initialized: now 1786387723983287 us; error 43 us; skew 500 ppm
Aug 10 18:48:43 dist-test-slave-cbsf krb5kdc[25994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387723, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.168.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:43.985170 26687 init.cc:377] Logged in from keytab as oryx/127.18.168.188@KRBTEST.COM (short username oryx)
I20260810 18:48:43.985692 26687 webserver.cc:460] Webserver started at http://127.18.168.188:42085/ using document root <none> and password file <none>
I20260810 18:48:43.985891 26687 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:43.985939 26687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:43.987399 26687 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 18:48:43.988502 26916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:43.988824 26687 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260810 18:48:43.988909 26687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-2/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-2/wal
uuid: "c8aece922db748f48528a46f14ec0d88"
format_stamp: "Formatted at 2026-08-10 18:48:43 on dist-test-slave-cbsf"
server_key: "c196f86e7c92732ed13f7ae3f2f96dda"
server_key_iv: "22aab3bca0d0304b473cea66b79b11d0"
server_key_version: "encryptionkey@0"
I20260810 18:48:43.988998 26687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:44.013481 26687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:44.016142 26687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:44.024197 26687 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.188:46047
I20260810 18:48:44.025054 26687 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/master-2/data/info.pb
I20260810 18:48:44.025444 26972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.188:46047 every 8 connection(s)
I20260810 18:48:44.028343 26973 sys_catalog.cc:263] Verifying existing consensus state
I20260810 18:48:44.029805 26973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88: Bootstrap starting.
I20260810 18:48:44.042156 26973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:48:44.042438 26973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88: Bootstrap complete.
I20260810 18:48:44.042976 26973 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: false } } peers { permanent_uuid: "c8aece922db748f48528a46f14ec0d88" member_type: NON_VOTER last_known_addr { host: "127.18.168.188" port: 46047 } attrs { promote: true } }
I20260810 18:48:44.043133 26973 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c8aece922db748f48528a46f14ec0d88, State: Initialized, Role: LEARNER
I20260810 18:48:44.043252 26973 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: false } } peers { permanent_uuid: "c8aece922db748f48528a46f14ec0d88" member_type: NON_VOTER last_known_addr { host: "127.18.168.188" port: 46047 } attrs { promote: true } }
I20260810 18:48:44.043680 26977 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: false } } peers { permanent_uuid: "c8aece922db748f48528a46f14ec0d88" member_type: NON_VOTER last_known_addr { host: "127.18.168.188" port: 46047 } attrs { promote: true } } }
I20260810 18:48:44.043776 26973 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:48:44.043867 26977 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [sys.catalog]: This master's current role is: LEARNER
I20260810 18:48:44.047721 26991 catalog_manager.cc:1259] Loaded cluster ID: 4f874a85450b418499bc2b5946eac0c8
I20260810 18:48:44.047796 26991 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88: loading cluster ID for follower catalog manager: success
I20260810 18:48:44.049369 26991 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88: acquiring CA information for follower catalog manager: success
I20260810 18:48:44.050418 26991 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 18:48:44.065979 26890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:44.066226 26890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:44.066268 26890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:44.070318 26890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:44.070374 26890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:44.070436 26890 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.129
I20260810 18:48:44.075644 26890 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:37727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.129.keytab
--principal=oryx/127.18.168.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.129:42891
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.168.129
--webserver_port=44363
--tserver_master_addrs=127.18.168.190:43053,127.18.168.189:44715,127.18.168.188:46047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.129
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:44.076745 26890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:44.077792 26890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:44.087589 26997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:44.087625 27000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:44.087711 26998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:44.087837 26890 server_base.cc:1034] running on GCE node
I20260810 18:48:44.088266 26890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:44.088968 26890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:44.090255 26890 hybrid_clock.cc:648] HybridClock initialized: now 1786387724090228 us; error 39 us; skew 500 ppm
Aug 10 18:48:44 dist-test-slave-cbsf krb5kdc[25994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387724, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.168.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:44.093066 26890 init.cc:377] Logged in from keytab as oryx/127.18.168.129@KRBTEST.COM (short username oryx)
I20260810 18:48:44.093830 26890 webserver.cc:460] Webserver started at http://127.18.168.129:44363/ using document root <none> and password file <none>
I20260810 18:48:44.094394 26890 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:44.094463 26890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:44.098511 26890 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:48:44.100627 27011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:44.102029 26890 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:48:44.102126 26890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-0/wal
uuid: "7023e9dd2aaa481cbe00bfe83efac6ed"
format_stamp: "Formatted at 2026-08-10 18:48:42 on dist-test-slave-cbsf"
server_key: "3815e100d495fa3444dab51b3d7b7a7c"
server_key_iv: "5b6222a5219b727cd1ea9ac29f9f45d2"
server_key_version: "encryptionkey@0"
I20260810 18:48:44.102800 26890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:44.136756 26890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:44.141062 26890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:44.141328 26890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:44.142172 26890 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:44.143182 26890 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:44.143254 26890 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:44.143327 26890 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:44.143378 26890 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:44.162930 26890 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.129:42891
I20260810 18:48:44.163080 27137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.129:42891 every 8 connection(s)
I20260810 18:48:44.164127 26890 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
I20260810 18:48:44.173650 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 26890
I20260810 18:48:44.173851 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 26265
Aug 10 18:48:44 dist-test-slave-cbsf krb5kdc[25994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387724, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.168.129@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM
Aug 10 18:48:44 dist-test-slave-cbsf krb5kdc[25994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387724, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.168.129@KRBTEST.COM for oryx/127.18.168.189@KRBTEST.COM
Aug 10 18:48:44 dist-test-slave-cbsf krb5kdc[25994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387724, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.168.129@KRBTEST.COM for oryx/127.18.168.188@KRBTEST.COM
I20260810 18:48:44.186764 26088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:44.166467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:44265 (local address 127.18.168.190:43053)
0810 18:48:44.166632 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:44.166638 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:44.168436 (+ 1798us) server_negotiation.cc:407] Connection header received
0810 18:48:44.170178 (+ 1742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:44.170183 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:44.170247 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:44.170314 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:44.172094 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.172762 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.176092 (+ 3330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.176506 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.176648 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:44.181344 (+ 4696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:44.181348 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:44.181350 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:44.181377 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:44.183899 (+ 2522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:44.184658 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:44.184665 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:44.184669 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:44.184825 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:44.185331 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:44.185334 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:44.185336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:44.185583 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:44.185663 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:44.186468 (+ 805us) server_negotiation.cc:299] Negotiation successful
0810 18:48:44.186607 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 18:48:44.187552 27142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:44.166414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:53379 (local address 127.18.168.188:46047)
0810 18:48:44.167502 (+ 1088us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:44.167508 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:44.169226 (+ 1718us) server_negotiation.cc:407] Connection header received
0810 18:48:44.170132 (+ 906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:44.170136 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:44.170197 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:44.170383 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:44.171924 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.173140 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.178077 (+ 4937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.178417 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.178547 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:44.183026 (+ 4479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:44.183030 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:44.183033 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:44.183060 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:44.185608 (+ 2548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:44.186097 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:44.186101 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:44.186103 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:44.186191 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:44.186540 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:44.186543 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:44.186545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:44.186746 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:44.186847 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:44.187209 (+ 362us) server_negotiation.cc:299] Negotiation successful
0810 18:48:44.187339 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":979,"thread_start_us":135,"threads_started":1}
I20260810 18:48:44.188175 26865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:44.167415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:38521 (local address 127.18.168.189:44715)
0810 18:48:44.167558 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:44.167565 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:44.168888 (+ 1323us) server_negotiation.cc:407] Connection header received
0810 18:48:44.170179 (+ 1291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:44.170183 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:44.170249 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:44.170314 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:44.172068 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.172852 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.176040 (+ 3188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.176465 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.176559 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:44.182027 (+ 5468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:44.182031 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:44.182034 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:44.182067 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:44.184618 (+ 2551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:44.185887 (+ 1269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:44.185895 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:44.185900 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:44.186052 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:44.186598 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:44.186606 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:44.186611 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:44.186978 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:44.187091 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:44.187693 (+ 602us) server_negotiation.cc:299] Negotiation successful
0810 18:48:44.187882 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 18:48:44.188555 27144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:44.167178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:46047 (local address 127.18.168.129:53379)
0810 18:48:44.169024 (+ 1846us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:44.169036 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:44.169882 (+ 846us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:44.170611 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:44.170613 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:44.171060 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:44.171764 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:44.171781 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.173380 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.173396 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:44.174873 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:44.174887 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.178542 (+ 3655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.178546 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:44.178710 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:44.179592 (+ 882us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:44.179604 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:44.182832 (+ 3228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:44.185727 (+ 2895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:44.185730 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:44.185731 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:44.185966 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:44.186303 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:44.186308 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:44.186329 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:44.186418 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:44.186925 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:44.186927 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:44.187022 (+ 95us) client_negotiation.cc:769] Sending connection context
0810 18:48:44.187742 (+ 720us) client_negotiation.cc:240] Negotiation successful
0810 18:48:44.187880 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1700,"mutex_wait_us":420,"thread_start_us":306,"threads_started":1}
I20260810 18:48:44.188555 27143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:44.167177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43053 (local address 127.18.168.129:44265)
0810 18:48:44.168099 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:44.168178 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:44.169881 (+ 1703us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:44.170598 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:44.170609 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:44.171320 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:44.171920 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:44.171930 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.173041 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.173044 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:44.175894 (+ 2850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:44.175907 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.177000 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.177006 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:44.177154 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:44.178207 (+ 1053us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:44.178238 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:44.181171 (+ 2933us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:44.184015 (+ 2844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:44.184023 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:44.184040 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:44.184471 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:44.185011 (+ 540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:44.185015 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:44.185019 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:44.185206 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:44.185744 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:44.185750 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:44.186160 (+ 410us) client_negotiation.cc:769] Sending connection context
0810 18:48:44.187027 (+ 867us) client_negotiation.cc:240] Negotiation successful
0810 18:48:44.187367 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"mutex_wait_us":1640,"spinlock_wait_cycles":28032,"thread_start_us":384,"threads_started":1}
I20260810 18:48:44.188985 27145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:44.167178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.189:44715 (local address 127.18.168.129:38521)
0810 18:48:44.168599 (+ 1421us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:44.168615 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:44.169881 (+ 1266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:44.170598 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:44.170609 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:44.171320 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:44.171886 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:44.171898 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.173117 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.173120 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:44.175873 (+ 2753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:44.175887 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.176801 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.176807 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:44.177119 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:44.178371 (+ 1252us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:44.178387 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:44.181833 (+ 3446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:44.185257 (+ 3424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:44.185265 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:44.185270 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:44.185702 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:44.186234 (+ 532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:44.186241 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:44.186247 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:44.186419 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:44.187207 (+ 788us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:44.187213 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:44.187371 (+ 158us) client_negotiation.cc:769] Sending connection context
0810 18:48:44.188365 (+ 994us) client_negotiation.cc:240] Negotiation successful
0810 18:48:44.188602 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1328,"mutex_wait_us":1198,"spinlock_wait_cycles":23168,"thread_start_us":324,"threads_started":1}
I20260810 18:48:44.190485 27139 heartbeater.cc:347] Connected to a master server at 127.18.168.190:43053
I20260810 18:48:44.190727 27139 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:44.191341 27139 heartbeater.cc:511] Master 127.18.168.190:43053 requested a full tablet report, sending...
I20260810 18:48:44.191498 27140 heartbeater.cc:347] Connected to a master server at 127.18.168.189:44715
I20260810 18:48:44.191603 27140 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:44.191758 27140 heartbeater.cc:511] Master 127.18.168.189:44715 requested a full tablet report, sending...
I20260810 18:48:44.190491 27138 heartbeater.cc:347] Connected to a master server at 127.18.168.188:46047
I20260810 18:48:44.192009 27138 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:44.192282 27138 heartbeater.cc:511] Master 127.18.168.188:46047 requested a full tablet report, sending...
I20260810 18:48:44.192984 26046 ts_manager.cc:194] Re-registered known tserver with Master: 7023e9dd2aaa481cbe00bfe83efac6ed (127.18.168.129:42891)
I20260810 18:48:44.193156 26726 ts_manager.cc:194] Registered new tserver with Master: 7023e9dd2aaa481cbe00bfe83efac6ed (127.18.168.129:42891)
I20260810 18:48:44.193459 26933 ts_manager.cc:194] Registered new tserver with Master: 7023e9dd2aaa481cbe00bfe83efac6ed (127.18.168.129:42891)
I20260810 18:48:44.193826 26046 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.168.129@KRBTEST.COM'} at 127.18.168.129:44265
I20260810 18:48:44.195127 26564 heartbeater.cc:503] Master 127.18.168.190:43053 was elected leader, sending a full tablet report...
I20260810 18:48:44.197718 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.130:41981
--local_ip_for_outbound_sockets=127.18.168.130
--tserver_master_addrs=127.18.168.190:43053
--webserver_port=39437
--webserver_interface=127.18.168.130
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.130.keytab
--principal=oryx/127.18.168.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:37727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.168.190:43053,127.18.168.189:44715
--tserver_master_addrs=127.18.168.190:43053,127.18.168.189:44715,127.18.168.188:46047 with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:44.315331 27146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:44.315587 27146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:44.315661 27146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:44.319746 27146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:44.319811 27146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:44.319939 27146 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.130
I20260810 18:48:44.325147 27146 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:37727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.130.keytab
--principal=oryx/127.18.168.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.130:41981
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.168.130
--webserver_port=39437
--tserver_master_addrs=127.18.168.190:43053,127.18.168.189:44715,127.18.168.188:46047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.130
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:44.326121 27146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:44.327109 27146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:44.337085 27155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:44.337211 27152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:44.337085 27153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:44.337597 27146 server_base.cc:1034] running on GCE node
I20260810 18:48:44.337982 27146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:44.338534 27146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:44.339761 27146 hybrid_clock.cc:648] HybridClock initialized: now 1786387724339736 us; error 31 us; skew 500 ppm
Aug 10 18:48:44 dist-test-slave-cbsf krb5kdc[25994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387724, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.168.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:44.342374 27146 init.cc:377] Logged in from keytab as oryx/127.18.168.130@KRBTEST.COM (short username oryx)
I20260810 18:48:44.343251 27146 webserver.cc:460] Webserver started at http://127.18.168.130:39437/ using document root <none> and password file <none>
I20260810 18:48:44.343881 27146 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:44.343997 27146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:44.347985 27146 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260810 18:48:44.350082 27163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:44.351303 27146 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:48:44.351435 27146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-1/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-1/wal
uuid: "59c9f2d08b94401989fdc32d65539feb"
format_stamp: "Formatted at 2026-08-10 18:48:42 on dist-test-slave-cbsf"
server_key: "420784fd2fb49bd4348fba68df1008aa"
server_key_iv: "bded2997401be0ae2150ea84d0183628"
server_key_version: "encryptionkey@0"
I20260810 18:48:44.351960 27146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:44.366441 27146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:44.369234 27146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:44.369426 27146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:44.369994 27146 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:44.370844 27146 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:44.370934 27146 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:44.371011 27146 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:44.371084 27146 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:44.379935 27146 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.130:41981
I20260810 18:48:44.379959 27290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.130:41981 every 8 connection(s)
I20260810 18:48:44.380960 27146 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
I20260810 18:48:44.384989 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 27146
I20260810 18:48:44.385196 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 26418
Aug 10 18:48:44 dist-test-slave-cbsf krb5kdc[25994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387724, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.168.130@KRBTEST.COM for oryx/127.18.168.189@KRBTEST.COM
Aug 10 18:48:44 dist-test-slave-cbsf krb5kdc[25994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387724, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.168.130@KRBTEST.COM for oryx/127.18.168.188@KRBTEST.COM
I20260810 18:48:44.402578 26865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:44.383336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:33855 (local address 127.18.168.189:44715)
0810 18:48:44.383481 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:44.383487 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:44.384596 (+ 1109us) server_negotiation.cc:407] Connection header received
0810 18:48:44.386443 (+ 1847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:44.386449 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:44.386512 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:44.386566 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:44.388766 (+ 2200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.389608 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.392375 (+ 2767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.392759 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.392842 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:44.397536 (+ 4694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:44.397540 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:44.397543 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:44.397573 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:44.399701 (+ 2128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:44.400446 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:44.400450 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:44.400452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:44.400557 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:44.401060 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:44.401064 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:44.401066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:44.401339 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:44.401418 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:44.402272 (+ 854us) server_negotiation.cc:299] Negotiation successful
0810 18:48:44.402406 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 18:48:44.402578 27142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:44.383096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:39053 (local address 127.18.168.188:46047)
0810 18:48:44.383242 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:44.383248 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:44.384389 (+ 1141us) server_negotiation.cc:407] Connection header received
0810 18:48:44.387707 (+ 3318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:44.387711 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:44.387768 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:44.387839 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:44.388738 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.389618 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.392375 (+ 2757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.392769 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.392842 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:44.397536 (+ 4694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:44.397540 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:44.397543 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:44.397573 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:44.399642 (+ 2069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:44.400446 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:44.400450 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:44.400452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:44.400557 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:44.401060 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:44.401064 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:44.401066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:44.401330 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:44.401435 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:44.402281 (+ 846us) server_negotiation.cc:299] Negotiation successful
0810 18:48:44.402408 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 18:48:44.404477 27298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:44.383201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:46047 (local address 127.18.168.130:39053)
0810 18:48:44.384246 (+ 1045us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:44.384259 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:44.385124 (+ 865us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:44.387982 (+ 2858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:44.387985 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:44.388213 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:44.388532 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:44.388554 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.389908 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.389921 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:44.392226 (+ 2305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:44.392237 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.393056 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.393060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:44.393369 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:44.394488 (+ 1119us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:44.394503 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:44.397323 (+ 2820us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:44.399793 (+ 2470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:44.399800 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:44.399818 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:44.400320 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:44.400672 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:44.400675 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:44.400677 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:44.400936 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:44.401512 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:44.401519 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:44.401945 (+ 426us) client_negotiation.cc:769] Sending connection context
0810 18:48:44.403172 (+ 1227us) client_negotiation.cc:240] Negotiation successful
0810 18:48:44.403285 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":965,"mutex_wait_us":449,"spinlock_wait_cycles":24576,"thread_start_us":153,"threads_started":1}
I20260810 18:48:44.404477 27299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:44.383201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.189:44715 (local address 127.18.168.130:33855)
0810 18:48:44.384507 (+ 1306us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:44.384519 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:44.385068 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:44.386967 (+ 1899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:44.386978 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:44.387625 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:44.388532 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:44.388554 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.389987 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.389990 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:44.392226 (+ 2236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:44.392237 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.393056 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.393060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:44.393371 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:44.394349 (+ 978us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:44.394380 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:44.397323 (+ 2943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:44.399809 (+ 2486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:44.399811 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:44.399818 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:44.400320 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:44.400680 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:44.400683 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:44.400685 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:44.400936 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:44.401512 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:44.401519 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:44.401945 (+ 426us) client_negotiation.cc:769] Sending connection context
0810 18:48:44.402838 (+ 893us) client_negotiation.cc:240] Negotiation successful
0810 18:48:44.403229 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1208,"mutex_wait_us":598,"spinlock_wait_cycles":57984,"thread_start_us":100,"threads_started":1}
I20260810 18:48:44.406267 27295 heartbeater.cc:347] Connected to a master server at 127.18.168.189:44715
I20260810 18:48:44.406533 27291 heartbeater.cc:347] Connected to a master server at 127.18.168.188:46047
I20260810 18:48:44.406533 27295 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:44.406680 27291 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:44.407311 27291 heartbeater.cc:511] Master 127.18.168.188:46047 requested a full tablet report, sending...
I20260810 18:48:44.407310 27295 heartbeater.cc:511] Master 127.18.168.189:44715 requested a full tablet report, sending...
I20260810 18:48:44.408659 26726 ts_manager.cc:194] Registered new tserver with Master: 59c9f2d08b94401989fdc32d65539feb (127.18.168.130:41981)
I20260810 18:48:44.408653 26933 ts_manager.cc:194] Registered new tserver with Master: 59c9f2d08b94401989fdc32d65539feb (127.18.168.130:41981)
Aug 10 18:48:44 dist-test-slave-cbsf krb5kdc[25994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387724, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.168.130@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM
I20260810 18:48:44.416146 26088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:44.382722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:54623 (local address 127.18.168.190:43053)
0810 18:48:44.382996 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:44.383003 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:44.383917 (+ 914us) server_negotiation.cc:407] Connection header received
0810 18:48:44.401891 (+ 17974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:44.401899 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:44.401995 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:44.402090 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:44.402952 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.404126 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.405755 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.406272 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.406387 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:44.412496 (+ 6109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:44.412499 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:44.412501 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:44.412527 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:44.414295 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:44.414880 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:44.414883 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:44.414884 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:44.414963 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:44.415283 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:44.415285 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:44.415287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:44.415460 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:44.415550 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:44.415907 (+ 357us) server_negotiation.cc:299] Negotiation successful
0810 18:48:44.416034 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169}
I20260810 18:48:44.416244 27297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:44.383201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43053 (local address 127.18.168.130:54623)
0810 18:48:44.383770 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:44.383820 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:44.385060 (+ 1240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:44.402308 (+ 17248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:44.402314 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:44.402580 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:44.402763 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:44.402777 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.404306 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.404313 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:44.405580 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:44.405595 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.408149 (+ 2554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.408157 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:44.408327 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:44.409758 (+ 1431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:44.409780 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:44.412338 (+ 2558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:44.414429 (+ 2091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:44.414432 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:44.414443 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:44.414763 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:44.415062 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:44.415068 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:44.415073 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:44.415181 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:44.415554 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:44.415559 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:44.415688 (+ 129us) client_negotiation.cc:769] Sending connection context
0810 18:48:44.415796 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 18:48:44.415966 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":166,"threads_started":1}
I20260810 18:48:44.417148 27293 heartbeater.cc:347] Connected to a master server at 127.18.168.190:43053
I20260810 18:48:44.417284 27293 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:44.417483 27293 heartbeater.cc:511] Master 127.18.168.190:43053 requested a full tablet report, sending...
I20260810 18:48:44.418151 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.131:45911
--local_ip_for_outbound_sockets=127.18.168.131
--tserver_master_addrs=127.18.168.190:43053
--webserver_port=45665
--webserver_interface=127.18.168.131
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.131.keytab
--principal=oryx/127.18.168.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:37727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.168.190:43053,127.18.168.189:44715
--tserver_master_addrs=127.18.168.190:43053,127.18.168.189:44715,127.18.168.188:46047 with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:48:44.418241 26046 ts_manager.cc:194] Re-registered known tserver with Master: 59c9f2d08b94401989fdc32d65539feb (127.18.168.130:41981)
I20260810 18:48:44.418820 26046 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.168.130@KRBTEST.COM'} at 127.18.168.130:54623
I20260810 18:48:44.526762 27142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:44.523172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47858 (local address 127.18.168.188:46047)
0810 18:48:44.523312 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:44.523318 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:44.523434 (+ 116us) server_negotiation.cc:407] Connection header received
0810 18:48:44.523599 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:44.523602 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:44.523658 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:44.523770 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:44.524064 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.524595 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.525653 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.526080 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.526241 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:44.526398 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:44.526484 (+ 86us) server_negotiation.cc:299] Negotiation successful
0810 18:48:44.526589 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 18:48:44.527168 27302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:44.523039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:46047 (local address 127.0.0.1:47858)
0810 18:48:44.523312 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:44.523329 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:44.523477 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:44.523784 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:44.523787 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:44.523809 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:44.523926 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:44.523933 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.524744 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.524747 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:44.525526 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:44.525535 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.526220 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.526241 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:44.526347 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:44.526372 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 18:48:44.526947 (+ 575us) client_negotiation.cc:240] Negotiation successful
0810 18:48:44.527050 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":91,"threads_started":1}
I20260810 18:48:44.527889 26945 raft_consensus.cc:1203] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260810 18:48:44.528264 26977 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1dde94f763cc421f8e7b1693c060e3f3. Latest consensus state: current_term: 1 leader_uuid: "1dde94f763cc421f8e7b1693c060e3f3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: false } } peers { permanent_uuid: "c8aece922db748f48528a46f14ec0d88" member_type: NON_VOTER last_known_addr { host: "127.18.168.188" port: 46047 } attrs { promote: true } } }
I20260810 18:48:44.528424 26977 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [sys.catalog]: This master's current role is: LEARNER
W20260810 18:48:44.531710 27300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:44.531975 27300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:44.532060 27300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:44.536304 27300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:44.536383 27300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:44.536500 27300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.131
I20260810 18:48:44.541625 27300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:37727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.131.keytab
--principal=oryx/127.18.168.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.131:45911
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.168.131
--webserver_port=45665
--tserver_master_addrs=127.18.168.190:43053,127.18.168.189:44715,127.18.168.188:46047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.131
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:44.542675 27300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:44.543690 27300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:44.553488 27309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:44.553565 27308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:44.553582 27311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:44.553925 27300 server_base.cc:1034] running on GCE node
I20260810 18:48:44.554298 27300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:44.554963 27300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:44.556146 27300 hybrid_clock.cc:648] HybridClock initialized: now 1786387724556126 us; error 43 us; skew 500 ppm
Aug 10 18:48:44 dist-test-slave-cbsf krb5kdc[25994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387724, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.168.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:44.558845 27300 init.cc:377] Logged in from keytab as oryx/127.18.168.131@KRBTEST.COM (short username oryx)
I20260810 18:48:44.559674 27300 webserver.cc:460] Webserver started at http://127.18.168.131:45665/ using document root <none> and password file <none>
I20260810 18:48:44.560220 27300 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:44.560326 27300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:44.564149 27300 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260810 18:48:44.566176 27319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:44.567292 27300 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260810 18:48:44.567427 27300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-2/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-2/wal
uuid: "aca3a4b0c4f8484c9a62e05683ce453e"
format_stamp: "Formatted at 2026-08-10 18:48:43 on dist-test-slave-cbsf"
server_key: "7b252ff7076d56ae61b11ca383ed0457"
server_key_iv: "99a0453cf8c07041c6587739ff0858df"
server_key_version: "encryptionkey@0"
I20260810 18:48:44.567953 27300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:44.594445 27300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:44.597467 27300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:44.597656 27300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:44.598285 27300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:44.599231 27300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:44.599287 27300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:44.599331 27300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:44.599361 27300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:44.612123 27300 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.131:45911
I20260810 18:48:44.612138 27445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.131:45911 every 8 connection(s)
I20260810 18:48:44.613164 27300 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
I20260810 18:48:44.615756 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 27300
Aug 10 18:48:44 dist-test-slave-cbsf krb5kdc[25994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387724, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.168.131@KRBTEST.COM for oryx/127.18.168.188@KRBTEST.COM
Aug 10 18:48:44 dist-test-slave-cbsf krb5kdc[25994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387724, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.168.131@KRBTEST.COM for oryx/127.18.168.189@KRBTEST.COM
I20260810 18:48:44.632124 27142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:44.616909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:46901 (local address 127.18.168.188:46047)
0810 18:48:44.617080 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:44.617087 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:44.617117 (+ 30us) server_negotiation.cc:407] Connection header received
0810 18:48:44.617948 (+ 831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:44.617954 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:44.618014 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:44.618072 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:44.620234 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.621069 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.622950 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.623294 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.623364 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:44.627737 (+ 4373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:44.627742 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:44.627746 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:44.627784 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:44.629515 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:44.630225 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:44.630229 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:44.630231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:44.630320 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:44.630771 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:44.630778 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:44.630782 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:44.631014 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:44.631107 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:44.631822 (+ 715us) server_negotiation.cc:299] Negotiation successful
0810 18:48:44.631943 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 18:48:44.633073 26865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:44.615281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:52811 (local address 127.18.168.189:44715)
0810 18:48:44.618021 (+ 2740us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:44.618026 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:44.618041 (+ 15us) server_negotiation.cc:407] Connection header received
0810 18:48:44.618084 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:44.618087 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:44.618131 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:44.618213 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:44.620235 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.621054 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.622950 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.623294 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.623364 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:44.628341 (+ 4977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:44.628345 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:44.628351 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:44.628379 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:44.630405 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:44.631239 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:44.631242 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:44.631244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:44.631325 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:44.631637 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:44.631640 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:44.631642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:44.631851 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:44.631920 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:44.632827 (+ 907us) server_negotiation.cc:299] Negotiation successful
0810 18:48:44.632939 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2636}
I20260810 18:48:44.633123 27462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:44.619513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52514 (local address 127.18.168.190:43053)
0810 18:48:44.622507 (+ 2994us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:44.622511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:44.623099 (+ 588us) server_negotiation.cc:407] Connection header received
0810 18:48:44.623142 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:44.623148 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:44.623199 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:44.623403 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:44.624875 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.625702 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.626891 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.627271 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.627362 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:44.628973 (+ 1611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:44.628976 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:44.628979 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:44.629011 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:44.631032 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:44.631688 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:44.631691 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:44.631693 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:44.631775 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:44.632045 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:44.632048 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:44.632049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:44.632228 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:44.632335 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:44.632782 (+ 447us) server_negotiation.cc:299] Negotiation successful
0810 18:48:44.632900 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2920,"thread_start_us":83,"threads_started":1}
Aug 10 18:48:44 dist-test-slave-cbsf krb5kdc[25994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387724, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.168.131@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM
I20260810 18:48:44.634588 27450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:44.615637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:46047 (local address 127.18.168.131:46901)
0810 18:48:44.616243 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:44.616295 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:44.617603 (+ 1308us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:44.618453 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:44.618462 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:44.619139 (+ 677us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:44.620059 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:44.620076 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.621222 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.621225 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:44.622808 (+ 1583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:44.622819 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.623551 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.623555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:44.623876 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:44.624574 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:44.624610 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:44.627507 (+ 2897us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:44.629715 (+ 2208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:44.629721 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:44.629735 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:44.630096 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:44.630445 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:44.630449 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:44.630452 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:44.630635 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:44.631212 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:44.631220 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:44.631537 (+ 317us) client_negotiation.cc:769] Sending connection context
0810 18:48:44.632587 (+ 1050us) client_negotiation.cc:240] Negotiation successful
0810 18:48:44.632947 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"mutex_wait_us":322,"spinlock_wait_cycles":32384,"thread_start_us":247,"threads_started":1}
I20260810 18:48:44.635141 27458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:44.615637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.189:44715 (local address 127.18.168.131:52811)
0810 18:48:44.617430 (+ 1793us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:44.617444 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:44.617677 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:44.618527 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:44.618530 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:44.619130 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:44.620056 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:44.620076 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.621222 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.621225 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:44.622808 (+ 1583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:44.622819 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.623551 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.623555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:44.624044 (+ 489us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:44.624919 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:44.624934 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:44.627962 (+ 3028us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:44.630500 (+ 2538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:44.630502 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:44.630504 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:44.630737 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:44.631447 (+ 710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:44.631449 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:44.631451 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:44.631531 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:44.632001 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:44.632004 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:44.632105 (+ 101us) client_negotiation.cc:769] Sending connection context
0810 18:48:44.633323 (+ 1218us) client_negotiation.cc:240] Negotiation successful
0810 18:48:44.633441 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1687,"mutex_wait_us":2042,"spinlock_wait_cycles":13696,"thread_start_us":450,"threads_started":1}
I20260810 18:48:44.636552 27448 heartbeater.cc:347] Connected to a master server at 127.18.168.189:44715
I20260810 18:48:44.636840 27448 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:44.637174 27446 heartbeater.cc:347] Connected to a master server at 127.18.168.188:46047
I20260810 18:48:44.637279 27446 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:44.637494 27448 heartbeater.cc:511] Master 127.18.168.189:44715 requested a full tablet report, sending...
I20260810 18:48:44.638649 26726 ts_manager.cc:194] Registered new tserver with Master: aca3a4b0c4f8484c9a62e05683ce453e (127.18.168.131:45911)
I20260810 18:48:44.637507 27446 heartbeater.cc:511] Master 127.18.168.188:46047 requested a full tablet report, sending...
I20260810 18:48:44.640394 26933 ts_manager.cc:194] Registered new tserver with Master: aca3a4b0c4f8484c9a62e05683ce453e (127.18.168.131:45911)
I20260810 18:48:44.637543 26046 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52514:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:48:44.642089 26046 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:48:44.642253 26088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:44.615436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:36771 (local address 127.18.168.190:43053)
0810 18:48:44.618447 (+ 3011us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:44.618453 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:44.618469 (+ 16us) server_negotiation.cc:407] Connection header received
0810 18:48:44.618528 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:44.618530 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:44.618579 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:44.618678 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:44.620241 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.621053 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.626432 (+ 5379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.626817 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.626901 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:44.638478 (+ 11577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:44.638481 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:44.638483 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:44.638506 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:44.640383 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:44.640968 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:44.640971 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:44.640973 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:44.641054 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:44.641362 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:44.641365 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:44.641367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:44.641576 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:44.641655 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:44.641992 (+ 337us) server_negotiation.cc:299] Negotiation successful
0810 18:48:44.642106 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2920}
I20260810 18:48:44.642655 27461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:44.618180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47866 (local address 127.18.168.188:46047)
0810 18:48:44.619184 (+ 1004us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:44.619190 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:44.619204 (+ 14us) server_negotiation.cc:407] Connection header received
0810 18:48:44.619246 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:44.619251 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:44.619297 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:44.620562 (+ 1265us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:44.620673 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.621386 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.622653 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.623090 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.623195 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:44.626911 (+ 3716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:44.626915 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:44.626918 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:44.626941 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:44.636617 (+ 9676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:44.638412 (+ 1795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:44.638418 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:44.638422 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:44.638509 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:44.638834 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:44.638839 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:44.638842 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:44.639089 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:44.641958 (+ 2869us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:44.642042 (+ 84us) server_negotiation.cc:299] Negotiation successful
0810 18:48:44.642249 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":935,"thread_start_us":177,"threads_started":1}
I20260810 18:48:44.642884 27459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:44.615637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43053 (local address 127.18.168.131:36771)
0810 18:48:44.617662 (+ 2025us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:44.617676 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:44.617912 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:44.618841 (+ 929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:44.618852 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:44.619733 (+ 881us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:44.620052 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:44.620076 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.621218 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.621220 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:44.626286 (+ 5066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:44.626301 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.630816 (+ 4515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.630825 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:44.630981 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:44.632368 (+ 1387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:44.632384 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:44.638151 (+ 5767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:44.640509 (+ 2358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:44.640512 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:44.640514 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:44.640843 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:44.641176 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:44.641178 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:44.641180 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:44.641262 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:44.641718 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:44.641720 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:44.641803 (+ 83us) client_negotiation.cc:769] Sending connection context
0810 18:48:44.642433 (+ 630us) client_negotiation.cc:240] Negotiation successful
0810 18:48:44.642607 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1944,"mutex_wait_us":5420,"thread_start_us":165,"threads_started":1}
I20260810 18:48:44.644542 27447 heartbeater.cc:347] Connected to a master server at 127.18.168.190:43053
I20260810 18:48:44.644636 27447 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:44.644892 27447 heartbeater.cc:511] Master 127.18.168.190:43053 requested a full tablet report, sending...
I20260810 18:48:44.645565 26045 ts_manager.cc:194] Re-registered known tserver with Master: aca3a4b0c4f8484c9a62e05683ce453e (127.18.168.131:45911)
I20260810 18:48:44.645931 27464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:44.621796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54070 (local address 127.18.168.189:44715)
0810 18:48:44.622651 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:44.622655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:44.623862 (+ 1207us) server_negotiation.cc:407] Connection header received
0810 18:48:44.623988 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:44.623991 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:44.624035 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:44.624250 (+ 215us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:44.625995 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.626772 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.636772 (+ 10000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.637201 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.637309 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:44.640295 (+ 2986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:44.640299 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:44.640302 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:44.640324 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:44.642644 (+ 2320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:44.643158 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:44.643161 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:44.643163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:44.643251 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:44.643528 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:44.643532 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:44.643545 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:44.643981 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:44.644123 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:44.644479 (+ 356us) server_negotiation.cc:299] Negotiation successful
0810 18:48:44.644600 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":773,"thread_start_us":74,"threads_started":1}
I20260810 18:48:44.645896 26681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1dde94f763cc421f8e7b1693c060e3f3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: false } } peers { permanent_uuid: "c8aece922db748f48528a46f14ec0d88" member_type: NON_VOTER last_known_addr { host: "127.18.168.188" port: 46047 } attrs { promote: true } } }
I20260810 18:48:44.646008 26681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:44.646266 26045 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.168.131@KRBTEST.COM'} at 127.18.168.131:36771
I20260810 18:48:44.649959 27467 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3: attempting to promote NON_VOTER c8aece922db748f48528a46f14ec0d88 to VOTER
I20260810 18:48:44.650290 27467 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: false } } peers { permanent_uuid: "c8aece922db748f48528a46f14ec0d88" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46047 } attrs { promote: false } }
I20260810 18:48:44.651496 26945 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [term 1 LEARNER]: Refusing update from remote peer 1dde94f763cc421f8e7b1693c060e3f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260810 18:48:44.651816 26737 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [term 1 FOLLOWER]: Refusing update from remote peer 1dde94f763cc421f8e7b1693c060e3f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260810 18:48:44.652251 27477 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8aece922db748f48528a46f14ec0d88" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46047 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260810 18:48:44.652421 27469 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260810 18:48:44.654248 27469 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, c8aece922db748f48528a46f14ec0d88 (127.18.168.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: false } } peers { permanent_uuid: "c8aece922db748f48528a46f14ec0d88" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46047 } attrs { promote: false } } }
I20260810 18:48:44.655228 26945 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, c8aece922db748f48528a46f14ec0d88 (127.18.168.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: false } } peers { permanent_uuid: "c8aece922db748f48528a46f14ec0d88" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46047 } attrs { promote: false } } }
I20260810 18:48:44.656292 27467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1dde94f763cc421f8e7b1693c060e3f3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: false } } peers { permanent_uuid: "c8aece922db748f48528a46f14ec0d88" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46047 } attrs { promote: false } } }
I20260810 18:48:44.656328 26681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1dde94f763cc421f8e7b1693c060e3f3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: false } } peers { permanent_uuid: "c8aece922db748f48528a46f14ec0d88" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46047 } attrs { promote: false } } }
I20260810 18:48:44.656414 27467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:44.656420 26681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:44.656534 27469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1dde94f763cc421f8e7b1693c060e3f3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: false } } peers { permanent_uuid: "c8aece922db748f48528a46f14ec0d88" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46047 } attrs { promote: false } } }
I20260810 18:48:44.656603 27469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:44.656494 26737 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, c8aece922db748f48528a46f14ec0d88 (127.18.168.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: false } } peers { permanent_uuid: "c8aece922db748f48528a46f14ec0d88" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46047 } attrs { promote: false } } }
I20260810 18:48:44.657526 26977 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1dde94f763cc421f8e7b1693c060e3f3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: false } } peers { permanent_uuid: "c8aece922db748f48528a46f14ec0d88" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46047 } attrs { promote: false } } }
I20260810 18:48:44.657642 26977 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:48:44.659180 27479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1dde94f763cc421f8e7b1693c060e3f3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: false } } peers { permanent_uuid: "c8aece922db748f48528a46f14ec0d88" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46047 } attrs { promote: false } } }
I20260810 18:48:44.659329 27479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:48:44.666137 27302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:44.660145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:42891 (local address 127.0.0.1:41666)
0810 18:48:44.660247 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:44.660264 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:44.660348 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:44.661574 (+ 1226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:44.661577 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:44.661597 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:44.661702 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:44.661709 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.663383 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.663388 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:44.664573 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:44.664584 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.665727 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.665731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:44.665824 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:44.665859 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 18:48:44.665931 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 18:48:44.666013 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260810 18:48:44.666500 27480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:44.660465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41666 (local address 127.18.168.129:42891)
0810 18:48:44.660788 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:44.660803 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:44.660831 (+ 28us) server_negotiation.cc:407] Connection header received
0810 18:48:44.660920 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:44.660926 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:44.661171 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:44.661864 (+ 693us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:44.662029 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.663266 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.664735 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.665603 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.665729 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:44.665861 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:44.665956 (+ 95us) server_negotiation.cc:299] Negotiation successful
0810 18:48:44.666165 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":83,"threads_started":1}
I20260810 18:48:44.666980 27482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:44.660762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.131:45911 (local address 127.0.0.1:60134)
0810 18:48:44.661454 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:44.661469 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:44.661719 (+ 250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:44.662504 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:44.662506 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:44.662523 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:44.662606 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:44.662611 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.663956 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.663959 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:44.665329 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:44.665341 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.666541 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.666546 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:44.666671 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:44.666696 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 18:48:44.666772 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 18:48:44.666841 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"mutex_wait_us":1,"thread_start_us":106,"threads_started":1}
I20260810 18:48:44.667327 27481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:44.661042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:41981 (local address 127.0.0.1:58802)
0810 18:48:44.662739 (+ 1697us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:44.662753 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:44.662957 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:44.663566 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:44.663569 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:44.663588 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:44.663661 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:44.663667 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.665000 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.665006 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:44.665759 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:44.665767 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.666916 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.666919 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:44.667026 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:44.667048 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 18:48:44.667121 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 18:48:44.667198 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1636,"thread_start_us":89,"threads_started":1}
I20260810 18:48:44.667858 27483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:44.661161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60134 (local address 127.18.168.131:45911)
0810 18:48:44.661829 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:44.661842 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:44.661861 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:48:44.661928 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:44.661933 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:44.662221 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:44.662521 (+ 300us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:44.662746 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.663830 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.665477 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.666239 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.666357 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:44.666468 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:44.667641 (+ 1173us) server_negotiation.cc:299] Negotiation successful
0810 18:48:44.667752 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":115,"threads_started":1}
I20260810 18:48:44.669059 27484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:44.661399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58802 (local address 127.18.168.130:41981)
0810 18:48:44.661913 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:44.661922 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:44.662965 (+ 1043us) server_negotiation.cc:407] Connection header received
0810 18:48:44.663054 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:44.663059 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:44.663298 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:44.663581 (+ 283us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:44.663793 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.664893 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.665895 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.666643 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.666758 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:44.666861 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:44.668861 (+ 2000us) server_negotiation.cc:299] Negotiation successful
0810 18:48:44.668943 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":190,"threads_started":1}
I20260810 18:48:44.669819 27062 tablet_service.cc:1467] Processing CreateTablet for tablet ae80b6a68c02430bb2f000df33431659 (DEFAULT_TABLE table=test-table [id=6348f4cc68c5432697a80158dd2ca1ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:44.670358 27365 tablet_service.cc:1467] Processing CreateTablet for tablet ae80b6a68c02430bb2f000df33431659 (DEFAULT_TABLE table=test-table [id=6348f4cc68c5432697a80158dd2ca1ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:44.671000 27062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae80b6a68c02430bb2f000df33431659. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:44.671325 27365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae80b6a68c02430bb2f000df33431659. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:44.671219 27215 tablet_service.cc:1467] Processing CreateTablet for tablet ae80b6a68c02430bb2f000df33431659 (DEFAULT_TABLE table=test-table [id=6348f4cc68c5432697a80158dd2ca1ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:44.672171 27215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae80b6a68c02430bb2f000df33431659. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:44.679277 27485 tablet_bootstrap.cc:492] T ae80b6a68c02430bb2f000df33431659 P 59c9f2d08b94401989fdc32d65539feb: Bootstrap starting.
I20260810 18:48:44.680776 27487 tablet_bootstrap.cc:492] T ae80b6a68c02430bb2f000df33431659 P 7023e9dd2aaa481cbe00bfe83efac6ed: Bootstrap starting.
I20260810 18:48:44.681049 27486 tablet_bootstrap.cc:492] T ae80b6a68c02430bb2f000df33431659 P aca3a4b0c4f8484c9a62e05683ce453e: Bootstrap starting.
I20260810 18:48:44.682339 27485 tablet_bootstrap.cc:654] T ae80b6a68c02430bb2f000df33431659 P 59c9f2d08b94401989fdc32d65539feb: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:44.683398 27486 tablet_bootstrap.cc:654] T ae80b6a68c02430bb2f000df33431659 P aca3a4b0c4f8484c9a62e05683ce453e: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:44.683398 27487 tablet_bootstrap.cc:654] T ae80b6a68c02430bb2f000df33431659 P 7023e9dd2aaa481cbe00bfe83efac6ed: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:44.683830 27485 log.cc:824] T ae80b6a68c02430bb2f000df33431659 P 59c9f2d08b94401989fdc32d65539feb: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:44.684376 27487 log.cc:824] T ae80b6a68c02430bb2f000df33431659 P 7023e9dd2aaa481cbe00bfe83efac6ed: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:44.684376 27486 log.cc:824] T ae80b6a68c02430bb2f000df33431659 P aca3a4b0c4f8484c9a62e05683ce453e: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:44.686127 27487 tablet_bootstrap.cc:492] T ae80b6a68c02430bb2f000df33431659 P 7023e9dd2aaa481cbe00bfe83efac6ed: No bootstrap required, opened a new log
I20260810 18:48:44.686229 27485 tablet_bootstrap.cc:492] T ae80b6a68c02430bb2f000df33431659 P 59c9f2d08b94401989fdc32d65539feb: No bootstrap required, opened a new log
I20260810 18:48:44.686362 27487 ts_tablet_manager.cc:1397] T ae80b6a68c02430bb2f000df33431659 P 7023e9dd2aaa481cbe00bfe83efac6ed: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 18:48:44.686409 27485 ts_tablet_manager.cc:1397] T ae80b6a68c02430bb2f000df33431659 P 59c9f2d08b94401989fdc32d65539feb: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260810 18:48:44.686610 27486 tablet_bootstrap.cc:492] T ae80b6a68c02430bb2f000df33431659 P aca3a4b0c4f8484c9a62e05683ce453e: No bootstrap required, opened a new log
I20260810 18:48:44.686873 27486 ts_tablet_manager.cc:1397] T ae80b6a68c02430bb2f000df33431659 P aca3a4b0c4f8484c9a62e05683ce453e: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260810 18:48:44.691134 27487 raft_consensus.cc:348] T ae80b6a68c02430bb2f000df33431659 P 7023e9dd2aaa481cbe00bfe83efac6ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7023e9dd2aaa481cbe00bfe83efac6ed" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 42891 } } peers { permanent_uuid: "aca3a4b0c4f8484c9a62e05683ce453e" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45911 } } peers { permanent_uuid: "59c9f2d08b94401989fdc32d65539feb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 41981 } }
I20260810 18:48:44.691412 27487 raft_consensus.cc:374] T ae80b6a68c02430bb2f000df33431659 P 7023e9dd2aaa481cbe00bfe83efac6ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:44.691470 27487 raft_consensus.cc:729] T ae80b6a68c02430bb2f000df33431659 P 7023e9dd2aaa481cbe00bfe83efac6ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7023e9dd2aaa481cbe00bfe83efac6ed, State: Initialized, Role: FOLLOWER
I20260810 18:48:44.691931 27487 consensus_queue.cc:260] T ae80b6a68c02430bb2f000df33431659 P 7023e9dd2aaa481cbe00bfe83efac6ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7023e9dd2aaa481cbe00bfe83efac6ed" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 42891 } } peers { permanent_uuid: "aca3a4b0c4f8484c9a62e05683ce453e" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45911 } } peers { permanent_uuid: "59c9f2d08b94401989fdc32d65539feb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 41981 } }
I20260810 18:48:44.693015 27487 ts_tablet_manager.cc:1428] T ae80b6a68c02430bb2f000df33431659 P 7023e9dd2aaa481cbe00bfe83efac6ed: Time spent starting tablet: real 0.007s user 0.004s sys 0.003s
I20260810 18:48:44.693295 27139 heartbeater.cc:503] Master 127.18.168.190:43053 was elected leader, sending a full tablet report...
I20260810 18:48:44.694613 27486 raft_consensus.cc:348] T ae80b6a68c02430bb2f000df33431659 P aca3a4b0c4f8484c9a62e05683ce453e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7023e9dd2aaa481cbe00bfe83efac6ed" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 42891 } } peers { permanent_uuid: "aca3a4b0c4f8484c9a62e05683ce453e" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45911 } } peers { permanent_uuid: "59c9f2d08b94401989fdc32d65539feb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 41981 } }
I20260810 18:48:44.694613 27485 raft_consensus.cc:348] T ae80b6a68c02430bb2f000df33431659 P 59c9f2d08b94401989fdc32d65539feb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7023e9dd2aaa481cbe00bfe83efac6ed" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 42891 } } peers { permanent_uuid: "aca3a4b0c4f8484c9a62e05683ce453e" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45911 } } peers { permanent_uuid: "59c9f2d08b94401989fdc32d65539feb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 41981 } }
I20260810 18:48:44.694860 27486 raft_consensus.cc:374] T ae80b6a68c02430bb2f000df33431659 P aca3a4b0c4f8484c9a62e05683ce453e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:44.694860 27485 raft_consensus.cc:374] T ae80b6a68c02430bb2f000df33431659 P 59c9f2d08b94401989fdc32d65539feb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:44.694917 27486 raft_consensus.cc:729] T ae80b6a68c02430bb2f000df33431659 P aca3a4b0c4f8484c9a62e05683ce453e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aca3a4b0c4f8484c9a62e05683ce453e, State: Initialized, Role: FOLLOWER
I20260810 18:48:44.694918 27485 raft_consensus.cc:729] T ae80b6a68c02430bb2f000df33431659 P 59c9f2d08b94401989fdc32d65539feb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59c9f2d08b94401989fdc32d65539feb, State: Initialized, Role: FOLLOWER
I20260810 18:48:44.695344 27485 consensus_queue.cc:260] T ae80b6a68c02430bb2f000df33431659 P 59c9f2d08b94401989fdc32d65539feb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7023e9dd2aaa481cbe00bfe83efac6ed" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 42891 } } peers { permanent_uuid: "aca3a4b0c4f8484c9a62e05683ce453e" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45911 } } peers { permanent_uuid: "59c9f2d08b94401989fdc32d65539feb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 41981 } }
I20260810 18:48:44.695344 27486 consensus_queue.cc:260] T ae80b6a68c02430bb2f000df33431659 P aca3a4b0c4f8484c9a62e05683ce453e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7023e9dd2aaa481cbe00bfe83efac6ed" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 42891 } } peers { permanent_uuid: "aca3a4b0c4f8484c9a62e05683ce453e" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45911 } } peers { permanent_uuid: "59c9f2d08b94401989fdc32d65539feb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 41981 } }
I20260810 18:48:44.696583 27486 ts_tablet_manager.cc:1428] T ae80b6a68c02430bb2f000df33431659 P aca3a4b0c4f8484c9a62e05683ce453e: Time spent starting tablet: real 0.010s user 0.005s sys 0.005s
I20260810 18:48:44.696600 27485 ts_tablet_manager.cc:1428] T ae80b6a68c02430bb2f000df33431659 P 59c9f2d08b94401989fdc32d65539feb: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260810 18:48:44.696894 27447 heartbeater.cc:503] Master 127.18.168.190:43053 was elected leader, sending a full tablet report...
I20260810 18:48:44.700286 27293 heartbeater.cc:503] Master 127.18.168.190:43053 was elected leader, sending a full tablet report...
I20260810 18:48:44.732816 27494 raft_consensus.cc:482] T ae80b6a68c02430bb2f000df33431659 P 59c9f2d08b94401989fdc32d65539feb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:48:44.733155 27494 raft_consensus.cc:504] T ae80b6a68c02430bb2f000df33431659 P 59c9f2d08b94401989fdc32d65539feb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7023e9dd2aaa481cbe00bfe83efac6ed" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 42891 } } peers { permanent_uuid: "aca3a4b0c4f8484c9a62e05683ce453e" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45911 } } peers { permanent_uuid: "59c9f2d08b94401989fdc32d65539feb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 41981 } }
I20260810 18:48:44.734591 27494 leader_election.cc:288] T ae80b6a68c02430bb2f000df33431659 P 59c9f2d08b94401989fdc32d65539feb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7023e9dd2aaa481cbe00bfe83efac6ed (127.18.168.129:42891), aca3a4b0c4f8484c9a62e05683ce453e (127.18.168.131:45911)
I20260810 18:48:44.738663 27483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:44.734982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:48157 (local address 127.18.168.131:45911)
0810 18:48:44.735101 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:44.735105 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:44.735371 (+ 266us) server_negotiation.cc:407] Connection header received
0810 18:48:44.735503 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:44.735505 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:44.735551 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:44.735631 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:44.736040 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.736528 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.737630 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.738028 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.738157 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:44.738214 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:44.738478 (+ 264us) server_negotiation.cc:299] Negotiation successful
0810 18:48:44.738550 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 18:48:44.738663 27298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:44.734889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.131:45911 (local address 127.18.168.130:48157)
0810 18:48:44.735243 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:44.735272 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:44.735374 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:44.735649 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:44.735652 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:44.735665 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:44.735879 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:44.735887 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.736649 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.736654 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:44.737499 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:44.737508 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.738147 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.738151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:44.738313 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:44.738334 (+ 21us) client_negotiation.cc:769] Sending connection context
0810 18:48:44.738447 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 18:48:44.738529 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":172}
I20260810 18:48:44.738842 27297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:44.734876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:42891 (local address 127.18.168.130:38731)
0810 18:48:44.735011 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:44.735042 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:44.735138 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:44.735498 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:44.735501 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:44.735521 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:44.735834 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:44.735842 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.736644 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.736650 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:44.737647 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:44.737658 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.738313 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.738317 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:44.738539 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:44.738556 (+ 17us) client_negotiation.cc:769] Sending connection context
0810 18:48:44.738649 (+ 93us) client_negotiation.cc:240] Negotiation successful
0810 18:48:44.738718 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":71}
I20260810 18:48:44.738842 27480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:44.734939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:38731 (local address 127.18.168.129:42891)
0810 18:48:44.735079 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:44.735083 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:44.735102 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:48:44.735334 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:44.735339 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:44.735389 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:44.735470 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:44.736005 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.736532 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.737789 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.738183 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.738313 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:44.738367 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:44.738669 (+ 302us) server_negotiation.cc:299] Negotiation successful
0810 18:48:44.738738 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 18:48:44.739584 27086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae80b6a68c02430bb2f000df33431659" candidate_uuid: "59c9f2d08b94401989fdc32d65539feb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7023e9dd2aaa481cbe00bfe83efac6ed" is_pre_election: true
I20260810 18:48:44.739634 27396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae80b6a68c02430bb2f000df33431659" candidate_uuid: "59c9f2d08b94401989fdc32d65539feb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aca3a4b0c4f8484c9a62e05683ce453e" is_pre_election: true
I20260810 18:48:44.739920 27086 raft_consensus.cc:2454] T ae80b6a68c02430bb2f000df33431659 P 7023e9dd2aaa481cbe00bfe83efac6ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59c9f2d08b94401989fdc32d65539feb in term 0.
I20260810 18:48:44.739957 27396 raft_consensus.cc:2454] T ae80b6a68c02430bb2f000df33431659 P aca3a4b0c4f8484c9a62e05683ce453e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59c9f2d08b94401989fdc32d65539feb in term 0.
I20260810 18:48:44.740561 27164 leader_election.cc:302] T ae80b6a68c02430bb2f000df33431659 P 59c9f2d08b94401989fdc32d65539feb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59c9f2d08b94401989fdc32d65539feb, 7023e9dd2aaa481cbe00bfe83efac6ed; no voters:
I20260810 18:48:44.740960 27494 raft_consensus.cc:2781] T ae80b6a68c02430bb2f000df33431659 P 59c9f2d08b94401989fdc32d65539feb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:48:44.741080 27494 raft_consensus.cc:482] T ae80b6a68c02430bb2f000df33431659 P 59c9f2d08b94401989fdc32d65539feb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:48:44.741142 27494 raft_consensus.cc:3037] T ae80b6a68c02430bb2f000df33431659 P 59c9f2d08b94401989fdc32d65539feb [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:44.742920 27494 raft_consensus.cc:504] T ae80b6a68c02430bb2f000df33431659 P 59c9f2d08b94401989fdc32d65539feb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7023e9dd2aaa481cbe00bfe83efac6ed" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 42891 } } peers { permanent_uuid: "aca3a4b0c4f8484c9a62e05683ce453e" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45911 } } peers { permanent_uuid: "59c9f2d08b94401989fdc32d65539feb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 41981 } }
I20260810 18:48:44.743252 27494 leader_election.cc:288] T ae80b6a68c02430bb2f000df33431659 P 59c9f2d08b94401989fdc32d65539feb [CANDIDATE]: Term 1 election: Requested vote from peers 7023e9dd2aaa481cbe00bfe83efac6ed (127.18.168.129:42891), aca3a4b0c4f8484c9a62e05683ce453e (127.18.168.131:45911)
I20260810 18:48:44.743633 27086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae80b6a68c02430bb2f000df33431659" candidate_uuid: "59c9f2d08b94401989fdc32d65539feb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7023e9dd2aaa481cbe00bfe83efac6ed"
I20260810 18:48:44.743660 27396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae80b6a68c02430bb2f000df33431659" candidate_uuid: "59c9f2d08b94401989fdc32d65539feb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aca3a4b0c4f8484c9a62e05683ce453e"
I20260810 18:48:44.743775 27086 raft_consensus.cc:3037] T ae80b6a68c02430bb2f000df33431659 P 7023e9dd2aaa481cbe00bfe83efac6ed [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:44.743813 27396 raft_consensus.cc:3037] T ae80b6a68c02430bb2f000df33431659 P aca3a4b0c4f8484c9a62e05683ce453e [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:44.745399 27086 raft_consensus.cc:2454] T ae80b6a68c02430bb2f000df33431659 P 7023e9dd2aaa481cbe00bfe83efac6ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59c9f2d08b94401989fdc32d65539feb in term 1.
I20260810 18:48:44.745664 27396 raft_consensus.cc:2454] T ae80b6a68c02430bb2f000df33431659 P aca3a4b0c4f8484c9a62e05683ce453e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59c9f2d08b94401989fdc32d65539feb in term 1.
I20260810 18:48:44.745698 27164 leader_election.cc:302] T ae80b6a68c02430bb2f000df33431659 P 59c9f2d08b94401989fdc32d65539feb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59c9f2d08b94401989fdc32d65539feb, 7023e9dd2aaa481cbe00bfe83efac6ed; no voters:
I20260810 18:48:44.745852 27494 raft_consensus.cc:2781] T ae80b6a68c02430bb2f000df33431659 P 59c9f2d08b94401989fdc32d65539feb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:44.746224 27494 raft_consensus.cc:686] T ae80b6a68c02430bb2f000df33431659 P 59c9f2d08b94401989fdc32d65539feb [term 1 LEADER]: Becoming Leader. State: Replica: 59c9f2d08b94401989fdc32d65539feb, State: Running, Role: LEADER
I20260810 18:48:44.746603 27494 consensus_queue.cc:237] T ae80b6a68c02430bb2f000df33431659 P 59c9f2d08b94401989fdc32d65539feb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7023e9dd2aaa481cbe00bfe83efac6ed" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 42891 } } peers { permanent_uuid: "aca3a4b0c4f8484c9a62e05683ce453e" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45911 } } peers { permanent_uuid: "59c9f2d08b94401989fdc32d65539feb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 41981 } }
I20260810 18:48:44.750340 26046 catalog_manager.cc:5495] T ae80b6a68c02430bb2f000df33431659 P 59c9f2d08b94401989fdc32d65539feb reported cstate change: term changed from 0 to 1, leader changed from <none> to 59c9f2d08b94401989fdc32d65539feb (127.18.168.130). New cstate: current_term: 1 leader_uuid: "59c9f2d08b94401989fdc32d65539feb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7023e9dd2aaa481cbe00bfe83efac6ed" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 42891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aca3a4b0c4f8484c9a62e05683ce453e" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59c9f2d08b94401989fdc32d65539feb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 41981 } health_report { overall_health: HEALTHY } } }
I20260810 18:48:44.769134 27484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:44.764793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58806 (local address 127.18.168.130:41981)
0810 18:48:44.764941 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:44.764945 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:44.765002 (+ 57us) server_negotiation.cc:407] Connection header received
0810 18:48:44.765212 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:44.765216 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:44.765292 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:44.765449 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:48:44.765776 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.766354 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.767243 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.767460 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.767548 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:44.768016 (+ 468us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:48:44.768514 (+ 498us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:48:44.768607 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:44.768888 (+ 281us) server_negotiation.cc:299] Negotiation successful
0810 18:48:44.768971 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 18:48:44.776328 27396 raft_consensus.cc:1261] T ae80b6a68c02430bb2f000df33431659 P aca3a4b0c4f8484c9a62e05683ce453e [term 1 FOLLOWER]: Refusing update from remote peer 59c9f2d08b94401989fdc32d65539feb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:44.776520 27086 raft_consensus.cc:1261] T ae80b6a68c02430bb2f000df33431659 P 7023e9dd2aaa481cbe00bfe83efac6ed [term 1 FOLLOWER]: Refusing update from remote peer 59c9f2d08b94401989fdc32d65539feb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:44.777251 27495 consensus_queue.cc:1035] T ae80b6a68c02430bb2f000df33431659 P 59c9f2d08b94401989fdc32d65539feb [LEADER]: Connected to new peer: Peer: permanent_uuid: "aca3a4b0c4f8484c9a62e05683ce453e" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:44.777498 27494 consensus_queue.cc:1035] T ae80b6a68c02430bb2f000df33431659 P 59c9f2d08b94401989fdc32d65539feb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7023e9dd2aaa481cbe00bfe83efac6ed" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 42891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:44.788878 27501 mvcc.cc:205] Tried to move back new op lower bound from 7317044120675311616 to 7317044120564871168. Current Snapshot: MvccSnapshot[applied={T|T < 7317044120675311616}]
I20260810 18:48:44.791445 27500 mvcc.cc:205] Tried to move back new op lower bound from 7317044120675311616 to 7317044120564871168. Current Snapshot: MvccSnapshot[applied={T|T < 7317044120675311616}]
I20260810 18:48:44.801793 26045 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:44.802065 26045 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:44.806707 26045 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dde94f763cc421f8e7b1693c060e3f3: Sending DeleteTablet for 3 replicas of tablet ae80b6a68c02430bb2f000df33431659
I20260810 18:48:44.807674 27062 tablet_service.cc:1514] Processing DeleteTablet for tablet ae80b6a68c02430bb2f000df33431659 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:44 UTC) from {username='oryx', principal='oryx/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:41666
I20260810 18:48:44.807852 27365 tablet_service.cc:1514] Processing DeleteTablet for tablet ae80b6a68c02430bb2f000df33431659 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:44 UTC) from {username='oryx', principal='oryx/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:60134
I20260810 18:48:44.808918 27509 tablet_replica.cc:330] T ae80b6a68c02430bb2f000df33431659 P aca3a4b0c4f8484c9a62e05683ce453e: stopping tablet replica
I20260810 18:48:44.808928 27508 tablet_replica.cc:330] T ae80b6a68c02430bb2f000df33431659 P 7023e9dd2aaa481cbe00bfe83efac6ed: stopping tablet replica
I20260810 18:48:44.809401 27508 raft_consensus.cc:2229] T ae80b6a68c02430bb2f000df33431659 P 7023e9dd2aaa481cbe00bfe83efac6ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:44.809326 27509 raft_consensus.cc:2229] T ae80b6a68c02430bb2f000df33431659 P aca3a4b0c4f8484c9a62e05683ce453e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:44.810037 27509 raft_consensus.cc:2258] T ae80b6a68c02430bb2f000df33431659 P aca3a4b0c4f8484c9a62e05683ce453e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:44.810091 27508 raft_consensus.cc:2258] T ae80b6a68c02430bb2f000df33431659 P 7023e9dd2aaa481cbe00bfe83efac6ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:44.811126 27215 tablet_service.cc:1514] Processing DeleteTablet for tablet ae80b6a68c02430bb2f000df33431659 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:44 UTC) from {username='oryx', principal='oryx/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:58802
I20260810 18:48:44.811980 27509 ts_tablet_manager.cc:1905] T ae80b6a68c02430bb2f000df33431659 P aca3a4b0c4f8484c9a62e05683ce453e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:44.812808 27508 ts_tablet_manager.cc:1905] T ae80b6a68c02430bb2f000df33431659 P 7023e9dd2aaa481cbe00bfe83efac6ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:44.813529 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 26010
I20260810 18:48:44.818362 27509 ts_tablet_manager.cc:1918] T ae80b6a68c02430bb2f000df33431659 P aca3a4b0c4f8484c9a62e05683ce453e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:48:44.818500 27509 log.cc:1196] T ae80b6a68c02430bb2f000df33431659 P aca3a4b0c4f8484c9a62e05683ce453e: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-2/wal/wals/ae80b6a68c02430bb2f000df33431659
I20260810 18:48:44.819018 27509 ts_tablet_manager.cc:1939] T ae80b6a68c02430bb2f000df33431659 P aca3a4b0c4f8484c9a62e05683ce453e: Deleting consensus metadata
I20260810 18:48:44.819882 27508 ts_tablet_manager.cc:1918] T ae80b6a68c02430bb2f000df33431659 P 7023e9dd2aaa481cbe00bfe83efac6ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:48:44.819991 27508 log.cc:1196] T ae80b6a68c02430bb2f000df33431659 P 7023e9dd2aaa481cbe00bfe83efac6ed: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-0/wal/wals/ae80b6a68c02430bb2f000df33431659
I20260810 18:48:44.820327 27508 ts_tablet_manager.cc:1939] T ae80b6a68c02430bb2f000df33431659 P 7023e9dd2aaa481cbe00bfe83efac6ed: Deleting consensus metadata
I20260810 18:48:44.820760 27511 tablet_replica.cc:330] T ae80b6a68c02430bb2f000df33431659 P 59c9f2d08b94401989fdc32d65539feb: stopping tablet replica
I20260810 18:48:44.821094 27511 raft_consensus.cc:2229] T ae80b6a68c02430bb2f000df33431659 P 59c9f2d08b94401989fdc32d65539feb [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:48:44.821537 27511 raft_consensus.cc:2258] T ae80b6a68c02430bb2f000df33431659 P 59c9f2d08b94401989fdc32d65539feb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:44.823081 27511 ts_tablet_manager.cc:1905] T ae80b6a68c02430bb2f000df33431659 P 59c9f2d08b94401989fdc32d65539feb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:44.826202 27511 ts_tablet_manager.cc:1918] T ae80b6a68c02430bb2f000df33431659 P 59c9f2d08b94401989fdc32d65539feb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:48:44.826305 27511 log.cc:1196] T ae80b6a68c02430bb2f000df33431659 P 59c9f2d08b94401989fdc32d65539feb: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-1/wal/wals/ae80b6a68c02430bb2f000df33431659
I20260810 18:48:44.826633 27511 ts_tablet_manager.cc:1939] T ae80b6a68c02430bb2f000df33431659 P 59c9f2d08b94401989fdc32d65539feb: Deleting consensus metadata
W20260810 18:48:44.826715 27320 connection.cc:535] server connection from 127.0.0.1:60134 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 18:48:44.826712 27015 connection.cc:535] server connection from 127.0.0.1:41666 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 18:48:44.827605 27167 connection.cc:441] server connection from 127.0.0.1:58802 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:58802 (request call id 1) could send its response
I20260810 18:48:44.838846 27464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:44.810320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54084 (local address 127.18.168.189:44715)
0810 18:48:44.810430 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:44.810434 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:44.827220 (+ 16786us) server_negotiation.cc:407] Connection header received
0810 18:48:44.827450 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:44.827453 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:44.827506 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:44.827561 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:44.829013 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.829704 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.830554 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.830957 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.831075 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:44.833586 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:44.833590 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:44.833592 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:44.833617 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:44.835374 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:44.836160 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:44.836171 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:44.836173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:44.836533 (+ 360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:44.837064 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:44.837067 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:44.837069 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:44.837388 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:44.838337 (+ 949us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:44.838440 (+ 103us) server_negotiation.cc:299] Negotiation successful
0810 18:48:44.838623 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 18:48:44.841156 26865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:44.831300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54092 (local address 127.18.168.189:44715)
0810 18:48:44.831415 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:44.831418 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:44.831617 (+ 199us) server_negotiation.cc:407] Connection header received
0810 18:48:44.831734 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:44.831737 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:44.831789 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:44.831883 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:44.832971 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.833696 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.835076 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.835424 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.835492 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:44.837312 (+ 1820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:44.837315 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:44.837318 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:44.837339 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:44.839216 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:44.839720 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:44.839723 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:44.839725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:44.839809 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:44.840104 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:44.840108 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:44.840110 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:44.840344 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:44.840482 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:44.840875 (+ 393us) server_negotiation.cc:299] Negotiation successful
0810 18:48:44.840988 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 18:48:44.842072 27142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:44.831241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47886 (local address 127.18.168.188:46047)
0810 18:48:44.831369 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:44.831375 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:44.831890 (+ 515us) server_negotiation.cc:407] Connection header received
0810 18:48:44.832050 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:44.832053 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:44.832103 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:44.832153 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:44.833390 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.834149 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.835355 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.835717 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.835789 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:44.837504 (+ 1715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:44.837508 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:44.837511 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:44.837536 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:44.840144 (+ 2608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:44.840613 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:44.840619 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:44.840620 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:44.840739 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:44.841072 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:44.841076 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:44.841078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:44.841293 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:44.841405 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:44.841795 (+ 390us) server_negotiation.cc:299] Negotiation successful
0810 18:48:44.841911 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 18:48:44.848785 27461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:44.809814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47876 (local address 127.18.168.188:46047)
0810 18:48:44.809961 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:44.809966 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:44.838383 (+ 28417us) server_negotiation.cc:407] Connection header received
0810 18:48:44.838546 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:44.838549 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:44.838608 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:44.838676 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:44.840526 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.841349 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.843112 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:44.843470 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:44.843597 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:44.845470 (+ 1873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:44.845474 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:44.845477 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:44.845502 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:44.847120 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:44.847596 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:44.847599 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:44.847601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:44.847680 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:44.847969 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:44.847973 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:44.847974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:44.848153 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:44.848252 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:44.848489 (+ 237us) server_negotiation.cc:299] Negotiation successful
0810 18:48:44.848584 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 18:48:45.792939 27520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:45.792349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43053 (local address 127.18.168.131:37411)
0810 18:48:45.792666 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:45.792786 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:43053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":96,"threads_started":1}
W20260810 18:48:45.793376 27447 heartbeater.cc:644] Failed to heartbeat to 127.18.168.190:43053 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.168.190:43053: connect: Connection refused (error 111)
I20260810 18:48:45.793432 27521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:45.792927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43053 (local address 127.18.168.130:54285)
0810 18:48:45.793234 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:45.793329 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:43053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":80,"threads_started":1}
W20260810 18:48:45.793776 27293 heartbeater.cc:644] Failed to heartbeat to 127.18.168.190:43053 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.168.190:43053: connect: Connection refused (error 111)
I20260810 18:48:45.794842 27522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:45.794478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43053 (local address 127.18.168.129:56045)
0810 18:48:45.794687 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:45.794767 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:43053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":74,"threads_started":1}
W20260810 18:48:45.795073 27139 heartbeater.cc:644] Failed to heartbeat to 127.18.168.190:43053 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.168.190:43053: connect: Connection refused (error 111)
I20260810 18:48:46.307664 27523 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dde94f763cc421f8e7b1693c060e3f3)
I20260810 18:48:46.307907 27523 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: false } } peers { permanent_uuid: "c8aece922db748f48528a46f14ec0d88" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46047 } attrs { promote: false } }
I20260810 18:48:46.309062 27523 leader_election.cc:288] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dde94f763cc421f8e7b1693c060e3f3 (127.18.168.190:43053), 397a954133b4468d90dcbc43ac570b0f (127.18.168.189:44715)
I20260810 18:48:46.309715 27524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:46.309242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43053 (local address 127.0.0.1:52992)
0810 18:48:46.309502 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:46.309590 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:43053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":62,"threads_started":1}
W20260810 18:48:46.309937 26920 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.168.190:43053: connect: Connection refused (error 111)
I20260810 18:48:46.310998 27524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:46.310716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43053 (local address 127.0.0.1:53006)
0810 18:48:46.310825 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:46.310904 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:43053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260810 18:48:46.311151 26920 leader_election.cc:334] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1dde94f763cc421f8e7b1693c060e3f3 (127.18.168.190:43053): Network error: Client connection negotiation failed: client connection to 127.18.168.190:43053: connect: Connection refused (error 111)
I20260810 18:48:46.311873 27528 raft_consensus.cc:482] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dde94f763cc421f8e7b1693c060e3f3)
I20260810 18:48:46.312065 27528 raft_consensus.cc:504] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: false } } peers { permanent_uuid: "c8aece922db748f48528a46f14ec0d88" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46047 } attrs { promote: false } }
I20260810 18:48:46.313201 27525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:46.309301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.189:44715 (local address 127.0.0.1:54094)
0810 18:48:46.309501 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:46.309518 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:46.309625 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:46.309935 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:46.309938 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:46.309961 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:46.310071 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:46.310079 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:46.310996 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:46.311000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:46.311926 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:46.311936 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:46.312804 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:46.312812 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:46.312885 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:46.312908 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 18:48:46.312972 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 18:48:46.313064 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":65,"threads_started":1}
I20260810 18:48:46.313259 27526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:46.309402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54094 (local address 127.18.168.189:44715)
0810 18:48:46.309664 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:46.309668 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:46.309683 (+ 15us) server_negotiation.cc:407] Connection header received
0810 18:48:46.309732 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:46.309734 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:46.309808 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:46.309892 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:46.310185 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:46.310891 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:46.312060 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:46.312654 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:46.312815 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:46.312869 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:46.313015 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 18:48:46.313100 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":73,"threads_started":1}
I20260810 18:48:46.313707 27529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:46.313191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43053 (local address 127.0.0.1:53008)
0810 18:48:46.313527 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:46.313613 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:43053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":60,"threads_started":1}
I20260810 18:48:46.313688 26737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c8aece922db748f48528a46f14ec0d88" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "397a954133b4468d90dcbc43ac570b0f" is_pre_election: true
I20260810 18:48:46.313784 27528 leader_election.cc:288] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dde94f763cc421f8e7b1693c060e3f3 (127.18.168.190:43053), c8aece922db748f48528a46f14ec0d88 (127.18.168.188:46047)
W20260810 18:48:46.313901 26710 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.168.190:43053: connect: Connection refused (error 111)
I20260810 18:48:46.313900 26737 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8aece922db748f48528a46f14ec0d88 in term 1.
I20260810 18:48:46.314420 26919 leader_election.cc:302] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 397a954133b4468d90dcbc43ac570b0f, c8aece922db748f48528a46f14ec0d88; no voters: 1dde94f763cc421f8e7b1693c060e3f3
I20260810 18:48:46.314657 27523 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 18:48:46.314734 27523 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1dde94f763cc421f8e7b1693c060e3f3)
I20260810 18:48:46.314783 27523 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [term 1 FOLLOWER]: Advancing to term 2
I20260810 18:48:46.316053 27523 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: false } } peers { permanent_uuid: "c8aece922db748f48528a46f14ec0d88" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46047 } attrs { promote: false } }
I20260810 18:48:46.316489 27523 leader_election.cc:288] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [CANDIDATE]: Term 2 election: Requested vote from peers 1dde94f763cc421f8e7b1693c060e3f3 (127.18.168.190:43053), 397a954133b4468d90dcbc43ac570b0f (127.18.168.189:44715)
I20260810 18:48:46.316788 27529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:46.314582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43053 (local address 127.0.0.1:53014)
0810 18:48:46.316651 (+ 2069us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:46.316720 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:43053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1997}
I20260810 18:48:46.316825 26737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c8aece922db748f48528a46f14ec0d88" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "397a954133b4468d90dcbc43ac570b0f"
W20260810 18:48:46.316882 26710 leader_election.cc:334] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1dde94f763cc421f8e7b1693c060e3f3 (127.18.168.190:43053): Network error: Client connection negotiation failed: client connection to 127.18.168.190:43053: connect: Connection refused (error 111)
I20260810 18:48:46.316921 26737 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [term 1 FOLLOWER]: Advancing to term 2
I20260810 18:48:46.316984 27525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:46.316589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43053 (local address 127.0.0.1:53028)
0810 18:48:46.316851 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:46.316905 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:43053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167}
I20260810 18:48:46.317744 27525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:46.317528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43053 (local address 127.0.0.1:53034)
0810 18:48:46.317636 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:46.317687 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:43053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260810 18:48:46.317852 26920 leader_election.cc:334] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1dde94f763cc421f8e7b1693c060e3f3 (127.18.168.190:43053): Network error: Client connection negotiation failed: client connection to 127.18.168.190:43053: connect: Connection refused (error 111)
I20260810 18:48:46.318167 26737 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8aece922db748f48528a46f14ec0d88 in term 2.
I20260810 18:48:46.318312 27530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:46.313247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:46047 (local address 127.0.0.1:47892)
0810 18:48:46.313977 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:46.313992 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:46.314176 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:46.314554 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:46.314557 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:46.314574 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:46.314655 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:46.314661 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:46.315601 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:46.315604 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:46.316761 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:46.316773 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:46.317691 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:46.317694 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:46.317784 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:46.317987 (+ 203us) client_negotiation.cc:769] Sending connection context
0810 18:48:46.318093 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 18:48:46.318166 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":668,"thread_start_us":87,"threads_started":1}
I20260810 18:48:46.318493 26919 leader_election.cc:302] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 397a954133b4468d90dcbc43ac570b0f, c8aece922db748f48528a46f14ec0d88; no voters: 1dde94f763cc421f8e7b1693c060e3f3
I20260810 18:48:46.318521 27531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:46.313632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47892 (local address 127.18.168.188:46047)
0810 18:48:46.313993 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:46.314003 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:46.314115 (+ 112us) server_negotiation.cc:407] Connection header received
0810 18:48:46.314306 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:46.314309 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:46.314355 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:46.314440 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:46.314786 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:46.315494 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:46.316927 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:46.317585 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:46.317696 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:46.317745 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:46.318109 (+ 364us) server_negotiation.cc:299] Negotiation successful
0810 18:48:46.318183 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":94,"threads_started":1}
I20260810 18:48:46.318723 27523 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 18:48:46.318945 27523 raft_consensus.cc:686] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [term 2 LEADER]: Becoming Leader. State: Replica: c8aece922db748f48528a46f14ec0d88, State: Running, Role: LEADER
I20260810 18:48:46.318959 26945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "397a954133b4468d90dcbc43ac570b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c8aece922db748f48528a46f14ec0d88" is_pre_election: true
I20260810 18:48:46.319227 27523 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: false } } peers { permanent_uuid: "c8aece922db748f48528a46f14ec0d88" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46047 } attrs { promote: false } }
I20260810 18:48:46.319483 26707 leader_election.cc:302] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 397a954133b4468d90dcbc43ac570b0f; no voters: 1dde94f763cc421f8e7b1693c060e3f3, c8aece922db748f48528a46f14ec0d88
I20260810 18:48:46.319675 27528 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 18:48:46.320670 27533 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8aece922db748f48528a46f14ec0d88. Latest consensus state: current_term: 2 leader_uuid: "c8aece922db748f48528a46f14ec0d88" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: false } } peers { permanent_uuid: "c8aece922db748f48528a46f14ec0d88" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46047 } attrs { promote: false } } }
I20260810 18:48:46.320830 27533 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:46.321148 27535 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:48:46.323514 27535 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:48:46.323922 27535 catalog_manager.cc:1259] Loaded cluster ID: 4f874a85450b418499bc2b5946eac0c8
I20260810 18:48:46.323971 27535 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:48:46.324493 27535 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:48:46.324952 27535 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88: Loaded TSK: 0
I20260810 18:48:46.325634 27535 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:48:46.356813 26932 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47886:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:48:46.359911 26932 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:48:46.362785 27525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:46.362554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43053 (local address 127.0.0.1:53040)
0810 18:48:46.362651 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:46.362714 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:43053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260810 18:48:46.362928 26737 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [term 2 FOLLOWER]: Refusing update from remote peer c8aece922db748f48528a46f14ec0d88: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260810 18:48:46.363504 27525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:46.363258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43053 (local address 127.0.0.1:53042)
0810 18:48:46.363366 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:46.363433 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:43053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260810 18:48:46.363544 27533 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260810 18:48:46.363729 26920 consensus_peers.cc:487] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 -> Peer 1dde94f763cc421f8e7b1693c060e3f3 (127.18.168.190:43053): Couldn't send request to peer 1dde94f763cc421f8e7b1693c060e3f3. Status: Network error: Client connection negotiation failed: client connection to 127.18.168.190:43053: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 18:48:46.365131 27528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8aece922db748f48528a46f14ec0d88. Latest consensus state: current_term: 2 leader_uuid: "c8aece922db748f48528a46f14ec0d88" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: false } } peers { permanent_uuid: "c8aece922db748f48528a46f14ec0d88" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46047 } attrs { promote: false } } }
I20260810 18:48:46.365262 27528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:48:46.366357 27523 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "c8aece922db748f48528a46f14ec0d88" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: false } } peers { permanent_uuid: "c8aece922db748f48528a46f14ec0d88" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46047 } attrs { promote: false } } }
I20260810 18:48:46.366472 27523 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:46.366931 27528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "c8aece922db748f48528a46f14ec0d88" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1dde94f763cc421f8e7b1693c060e3f3" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43053 } } peers { permanent_uuid: "397a954133b4468d90dcbc43ac570b0f" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 44715 } attrs { promote: false } } peers { permanent_uuid: "c8aece922db748f48528a46f14ec0d88" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46047 } attrs { promote: false } } }
I20260810 18:48:46.367069 27528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 397a954133b4468d90dcbc43ac570b0f [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:48:46.377339 27525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:46.372749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.131:45911 (local address 127.0.0.1:60146)
0810 18:48:46.373053 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:46.373072 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:46.373165 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:46.373613 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:46.373619 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:46.373643 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:46.373753 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:46.373763 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:46.374722 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:46.374726 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:46.375817 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:46.375825 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:46.376850 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:46.376854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:46.376961 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:46.376990 (+ 29us) client_negotiation.cc:769] Sending connection context
0810 18:48:46.377073 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 18:48:46.377210 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":54}
I20260810 18:48:46.377384 27546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:46.373029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60146 (local address 127.18.168.131:45911)
0810 18:48:46.373364 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:46.373370 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:46.373387 (+ 17us) server_negotiation.cc:407] Connection header received
0810 18:48:46.373448 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:46.373451 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:46.373511 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:46.373603 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:46.373899 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:46.374607 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:46.375985 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:46.376689 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:46.376869 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:46.376965 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:46.377109 (+ 144us) server_negotiation.cc:299] Negotiation successful
0810 18:48:46.377214 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":111,"threads_started":1}
I20260810 18:48:46.377739 27524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:46.373289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:42891 (local address 127.0.0.1:41670)
0810 18:48:46.373434 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:46.373449 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:46.373523 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:46.374098 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:46.374104 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:46.374127 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:46.374233 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:46.374239 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:46.375277 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:46.375280 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:46.376290 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:46.376318 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:46.377335 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:46.377338 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:46.377439 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:46.377468 (+ 29us) client_negotiation.cc:769] Sending connection context
0810 18:48:46.377535 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 18:48:46.377606 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260810 18:48:46.377938 27365 tablet_service.cc:1467] Processing CreateTablet for tablet c573413479ce429a8a03150ca410db1e (DEFAULT_TABLE table=test-table [id=5fbe1d74cc154db5ad9c7fb9b198b32f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:46.378254 27365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c573413479ce429a8a03150ca410db1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:46.378463 27547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:46.373557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41670 (local address 127.18.168.129:42891)
0810 18:48:46.373821 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:46.373826 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:46.373842 (+ 16us) server_negotiation.cc:407] Connection header received
0810 18:48:46.373901 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:46.373906 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:46.373959 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:46.374333 (+ 374us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:46.374471 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:46.375151 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:46.376463 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:46.377098 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:46.377228 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:46.377279 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:46.378245 (+ 966us) server_negotiation.cc:299] Negotiation successful
0810 18:48:46.378337 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":75,"threads_started":1}
I20260810 18:48:46.378995 27062 tablet_service.cc:1467] Processing CreateTablet for tablet c573413479ce429a8a03150ca410db1e (DEFAULT_TABLE table=test-table [id=5fbe1d74cc154db5ad9c7fb9b198b32f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:46.379287 27062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c573413479ce429a8a03150ca410db1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:46.379554 27549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:46.373858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:41981 (local address 127.0.0.1:58808)
0810 18:48:46.374204 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:46.374217 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:46.374296 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:46.375751 (+ 1455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:46.375755 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:46.375773 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:46.375867 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:46.375877 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:46.376981 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:46.376987 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:46.378152 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:46.378162 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:46.379125 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:46.379129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:46.379234 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:46.379265 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 18:48:46.379347 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 18:48:46.379429 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":78,"threads_started":1}
I20260810 18:48:46.380854 27553 tablet_bootstrap.cc:492] T c573413479ce429a8a03150ca410db1e P aca3a4b0c4f8484c9a62e05683ce453e: Bootstrap starting.
I20260810 18:48:46.381721 27553 tablet_bootstrap.cc:654] T c573413479ce429a8a03150ca410db1e P aca3a4b0c4f8484c9a62e05683ce453e: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:46.381983 27554 tablet_bootstrap.cc:492] T c573413479ce429a8a03150ca410db1e P 7023e9dd2aaa481cbe00bfe83efac6ed: Bootstrap starting.
I20260810 18:48:46.382977 27550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:46.374005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58808 (local address 127.18.168.130:41981)
0810 18:48:46.375420 (+ 1415us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:46.375425 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:46.375440 (+ 15us) server_negotiation.cc:407] Connection header received
0810 18:48:46.375495 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:46.375498 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:46.375548 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:46.375636 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:46.376008 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:46.376847 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:46.378337 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:46.378878 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:46.379002 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:46.379059 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:46.379725 (+ 666us) server_negotiation.cc:299] Negotiation successful
0810 18:48:46.379809 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1316,"thread_start_us":84,"threads_started":1}
I20260810 18:48:46.383227 27554 tablet_bootstrap.cc:654] T c573413479ce429a8a03150ca410db1e P 7023e9dd2aaa481cbe00bfe83efac6ed: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:46.383520 27553 tablet_bootstrap.cc:492] T c573413479ce429a8a03150ca410db1e P aca3a4b0c4f8484c9a62e05683ce453e: No bootstrap required, opened a new log
I20260810 18:48:46.383548 27215 tablet_service.cc:1467] Processing CreateTablet for tablet c573413479ce429a8a03150ca410db1e (DEFAULT_TABLE table=test-table [id=5fbe1d74cc154db5ad9c7fb9b198b32f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:46.383615 27553 ts_tablet_manager.cc:1397] T c573413479ce429a8a03150ca410db1e P aca3a4b0c4f8484c9a62e05683ce453e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 18:48:46.383837 27215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c573413479ce429a8a03150ca410db1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:46.384089 27553 raft_consensus.cc:348] T c573413479ce429a8a03150ca410db1e P aca3a4b0c4f8484c9a62e05683ce453e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aca3a4b0c4f8484c9a62e05683ce453e" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45911 } } peers { permanent_uuid: "7023e9dd2aaa481cbe00bfe83efac6ed" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 42891 } } peers { permanent_uuid: "59c9f2d08b94401989fdc32d65539feb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 41981 } }
I20260810 18:48:46.384222 27553 raft_consensus.cc:374] T c573413479ce429a8a03150ca410db1e P aca3a4b0c4f8484c9a62e05683ce453e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:46.384259 27553 raft_consensus.cc:729] T c573413479ce429a8a03150ca410db1e P aca3a4b0c4f8484c9a62e05683ce453e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aca3a4b0c4f8484c9a62e05683ce453e, State: Initialized, Role: FOLLOWER
I20260810 18:48:46.384449 27553 consensus_queue.cc:260] T c573413479ce429a8a03150ca410db1e P aca3a4b0c4f8484c9a62e05683ce453e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aca3a4b0c4f8484c9a62e05683ce453e" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45911 } } peers { permanent_uuid: "7023e9dd2aaa481cbe00bfe83efac6ed" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 42891 } } peers { permanent_uuid: "59c9f2d08b94401989fdc32d65539feb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 41981 } }
I20260810 18:48:46.384593 27554 tablet_bootstrap.cc:492] T c573413479ce429a8a03150ca410db1e P 7023e9dd2aaa481cbe00bfe83efac6ed: No bootstrap required, opened a new log
I20260810 18:48:46.384708 27554 ts_tablet_manager.cc:1397] T c573413479ce429a8a03150ca410db1e P 7023e9dd2aaa481cbe00bfe83efac6ed: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 18:48:46.384850 27553 ts_tablet_manager.cc:1428] T c573413479ce429a8a03150ca410db1e P aca3a4b0c4f8484c9a62e05683ce453e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 18:48:46.385748 27558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:46.385267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43053 (local address 127.18.168.131:41381)
0810 18:48:46.385575 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:46.385650 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:43053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":100,"threads_started":1}
I20260810 18:48:46.385870 27554 raft_consensus.cc:348] T c573413479ce429a8a03150ca410db1e P 7023e9dd2aaa481cbe00bfe83efac6ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aca3a4b0c4f8484c9a62e05683ce453e" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45911 } } peers { permanent_uuid: "7023e9dd2aaa481cbe00bfe83efac6ed" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 42891 } } peers { permanent_uuid: "59c9f2d08b94401989fdc32d65539feb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 41981 } }
I20260810 18:48:46.386263 27554 raft_consensus.cc:374] T c573413479ce429a8a03150ca410db1e P 7023e9dd2aaa481cbe00bfe83efac6ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:46.386304 27554 raft_consensus.cc:729] T c573413479ce429a8a03150ca410db1e P 7023e9dd2aaa481cbe00bfe83efac6ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7023e9dd2aaa481cbe00bfe83efac6ed, State: Initialized, Role: FOLLOWER
I20260810 18:48:46.386507 27554 consensus_queue.cc:260] T c573413479ce429a8a03150ca410db1e P 7023e9dd2aaa481cbe00bfe83efac6ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aca3a4b0c4f8484c9a62e05683ce453e" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45911 } } peers { permanent_uuid: "7023e9dd2aaa481cbe00bfe83efac6ed" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 42891 } } peers { permanent_uuid: "59c9f2d08b94401989fdc32d65539feb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 41981 } }
I20260810 18:48:46.386811 27560 tablet_bootstrap.cc:492] T c573413479ce429a8a03150ca410db1e P 59c9f2d08b94401989fdc32d65539feb: Bootstrap starting.
I20260810 18:48:46.386860 27554 ts_tablet_manager.cc:1428] T c573413479ce429a8a03150ca410db1e P 7023e9dd2aaa481cbe00bfe83efac6ed: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260810 18:48:46.387890 27560 tablet_bootstrap.cc:654] T c573413479ce429a8a03150ca410db1e P 59c9f2d08b94401989fdc32d65539feb: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:46.388494 27558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:46.386282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43053 (local address 127.18.168.131:46807)
0810 18:48:46.388377 (+ 2095us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:46.388433 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:43053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2029}
I20260810 18:48:46.388917 27562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:46.387834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43053 (local address 127.18.168.129:43815)
0810 18:48:46.388758 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:46.388814 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:43053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":818,"thread_start_us":91,"threads_started":1}
I20260810 18:48:46.389899 27446 heartbeater.cc:503] Master 127.18.168.188:46047 was elected leader, sending a full tablet report...
I20260810 18:48:46.390547 27560 tablet_bootstrap.cc:492] T c573413479ce429a8a03150ca410db1e P 59c9f2d08b94401989fdc32d65539feb: No bootstrap required, opened a new log
I20260810 18:48:46.390653 27560 ts_tablet_manager.cc:1397] T c573413479ce429a8a03150ca410db1e P 59c9f2d08b94401989fdc32d65539feb: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260810 18:48:46.391031 27560 raft_consensus.cc:348] T c573413479ce429a8a03150ca410db1e P 59c9f2d08b94401989fdc32d65539feb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aca3a4b0c4f8484c9a62e05683ce453e" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45911 } } peers { permanent_uuid: "7023e9dd2aaa481cbe00bfe83efac6ed" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 42891 } } peers { permanent_uuid: "59c9f2d08b94401989fdc32d65539feb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 41981 } }
I20260810 18:48:46.391141 27560 raft_consensus.cc:374] T c573413479ce429a8a03150ca410db1e P 59c9f2d08b94401989fdc32d65539feb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:46.391175 27560 raft_consensus.cc:729] T c573413479ce429a8a03150ca410db1e P 59c9f2d08b94401989fdc32d65539feb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59c9f2d08b94401989fdc32d65539feb, State: Initialized, Role: FOLLOWER
I20260810 18:48:46.391304 27560 consensus_queue.cc:260] T c573413479ce429a8a03150ca410db1e P 59c9f2d08b94401989fdc32d65539feb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aca3a4b0c4f8484c9a62e05683ce453e" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45911 } } peers { permanent_uuid: "7023e9dd2aaa481cbe00bfe83efac6ed" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 42891 } } peers { permanent_uuid: "59c9f2d08b94401989fdc32d65539feb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 41981 } }
I20260810 18:48:46.391568 27560 ts_tablet_manager.cc:1428] T c573413479ce429a8a03150ca410db1e P 59c9f2d08b94401989fdc32d65539feb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 18:48:46.392452 27565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:46.391881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43053 (local address 127.18.168.130:35777)
0810 18:48:46.392292 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:46.392351 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:43053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":90,"threads_started":1}
I20260810 18:48:46.393004 27291 heartbeater.cc:503] Master 127.18.168.188:46047 was elected leader, sending a full tablet report...
I20260810 18:48:46.393157 27565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:46.392899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43053 (local address 127.18.168.130:34059)
0810 18:48:46.393036 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:46.393098 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:43053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260810 18:48:46.417578 27141 tablet.cc:2369] T c573413479ce429a8a03150ca410db1e P 7023e9dd2aaa481cbe00bfe83efac6ed: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:48:46.448907 27557 raft_consensus.cc:482] T c573413479ce429a8a03150ca410db1e P aca3a4b0c4f8484c9a62e05683ce453e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:48:46.449062 27557 raft_consensus.cc:504] T c573413479ce429a8a03150ca410db1e P aca3a4b0c4f8484c9a62e05683ce453e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aca3a4b0c4f8484c9a62e05683ce453e" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45911 } } peers { permanent_uuid: "7023e9dd2aaa481cbe00bfe83efac6ed" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 42891 } } peers { permanent_uuid: "59c9f2d08b94401989fdc32d65539feb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 41981 } }
I20260810 18:48:46.450155 27557 leader_election.cc:288] T c573413479ce429a8a03150ca410db1e P aca3a4b0c4f8484c9a62e05683ce453e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7023e9dd2aaa481cbe00bfe83efac6ed (127.18.168.129:42891), 59c9f2d08b94401989fdc32d65539feb (127.18.168.130:41981)
I20260810 18:48:46.453871 27558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:46.450391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:42891 (local address 127.18.168.131:46263)
0810 18:48:46.450562 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:46.450580 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:46.450698 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:46.450979 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:46.450982 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:46.451003 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:46.451137 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:46.451142 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:46.451890 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:46.451895 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:46.452752 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:46.452762 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:46.453432 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:46.453436 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:46.453534 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:46.453561 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 18:48:46.453657 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 18:48:46.453726 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260810 18:48:46.453871 27547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:46.450555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:46263 (local address 127.18.168.129:42891)
0810 18:48:46.450729 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:46.450734 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:46.450750 (+ 16us) server_negotiation.cc:407] Connection header received
0810 18:48:46.450810 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:46.450813 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:46.450864 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:46.450953 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:46.451262 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:46.451773 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:46.452885 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:46.453293 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:46.453437 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:46.453491 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:46.453667 (+ 176us) server_negotiation.cc:299] Negotiation successful
0810 18:48:46.453741 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 18:48:46.454164 27550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:46.450554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:50589 (local address 127.18.168.130:41981)
0810 18:48:46.450730 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:46.450738 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:46.451089 (+ 351us) server_negotiation.cc:407] Connection header received
0810 18:48:46.451241 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:46.451244 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:46.451294 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:46.451374 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:46.451627 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:46.452120 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:46.453190 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:46.453609 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:46.453730 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:46.453778 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:46.453976 (+ 198us) server_negotiation.cc:299] Negotiation successful
0810 18:48:46.454048 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 18:48:46.454174 27566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:46.450391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:41981 (local address 127.18.168.131:50589)
0810 18:48:46.450988 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:46.451000 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:46.451124 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:46.451385 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:46.451388 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:46.451403 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:46.451514 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:46.451519 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:46.452254 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:46.452259 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:46.453068 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:46.453077 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:46.453730 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:46.453733 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:46.453834 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:46.453868 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 18:48:46.453960 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 18:48:46.454037 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"mutex_wait_us":245,"thread_start_us":174,"threads_started":1}
I20260810 18:48:46.454332 27086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c573413479ce429a8a03150ca410db1e" candidate_uuid: "aca3a4b0c4f8484c9a62e05683ce453e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7023e9dd2aaa481cbe00bfe83efac6ed" is_pre_election: true
I20260810 18:48:46.454496 27086 raft_consensus.cc:2454] T c573413479ce429a8a03150ca410db1e P 7023e9dd2aaa481cbe00bfe83efac6ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aca3a4b0c4f8484c9a62e05683ce453e in term 0.
I20260810 18:48:46.454728 27239 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c573413479ce429a8a03150ca410db1e" candidate_uuid: "aca3a4b0c4f8484c9a62e05683ce453e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59c9f2d08b94401989fdc32d65539feb" is_pre_election: true
I20260810 18:48:46.454941 27320 leader_election.cc:302] T c573413479ce429a8a03150ca410db1e P aca3a4b0c4f8484c9a62e05683ce453e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7023e9dd2aaa481cbe00bfe83efac6ed, aca3a4b0c4f8484c9a62e05683ce453e; no voters:
I20260810 18:48:46.455005 27239 raft_consensus.cc:2454] T c573413479ce429a8a03150ca410db1e P 59c9f2d08b94401989fdc32d65539feb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aca3a4b0c4f8484c9a62e05683ce453e in term 0.
I20260810 18:48:46.455227 27557 raft_consensus.cc:2781] T c573413479ce429a8a03150ca410db1e P aca3a4b0c4f8484c9a62e05683ce453e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:48:46.455307 27557 raft_consensus.cc:482] T c573413479ce429a8a03150ca410db1e P aca3a4b0c4f8484c9a62e05683ce453e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:48:46.455380 27557 raft_consensus.cc:3037] T c573413479ce429a8a03150ca410db1e P aca3a4b0c4f8484c9a62e05683ce453e [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:46.456467 27557 raft_consensus.cc:504] T c573413479ce429a8a03150ca410db1e P aca3a4b0c4f8484c9a62e05683ce453e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aca3a4b0c4f8484c9a62e05683ce453e" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45911 } } peers { permanent_uuid: "7023e9dd2aaa481cbe00bfe83efac6ed" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 42891 } } peers { permanent_uuid: "59c9f2d08b94401989fdc32d65539feb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 41981 } }
I20260810 18:48:46.456888 27557 leader_election.cc:288] T c573413479ce429a8a03150ca410db1e P aca3a4b0c4f8484c9a62e05683ce453e [CANDIDATE]: Term 1 election: Requested vote from peers 7023e9dd2aaa481cbe00bfe83efac6ed (127.18.168.129:42891), 59c9f2d08b94401989fdc32d65539feb (127.18.168.130:41981)
I20260810 18:48:46.457233 27086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c573413479ce429a8a03150ca410db1e" candidate_uuid: "aca3a4b0c4f8484c9a62e05683ce453e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7023e9dd2aaa481cbe00bfe83efac6ed"
I20260810 18:48:46.457284 27239 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c573413479ce429a8a03150ca410db1e" candidate_uuid: "aca3a4b0c4f8484c9a62e05683ce453e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59c9f2d08b94401989fdc32d65539feb"
I20260810 18:48:46.457374 27086 raft_consensus.cc:3037] T c573413479ce429a8a03150ca410db1e P 7023e9dd2aaa481cbe00bfe83efac6ed [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:46.457408 27239 raft_consensus.cc:3037] T c573413479ce429a8a03150ca410db1e P 59c9f2d08b94401989fdc32d65539feb [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:46.458318 27239 raft_consensus.cc:2454] T c573413479ce429a8a03150ca410db1e P 59c9f2d08b94401989fdc32d65539feb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aca3a4b0c4f8484c9a62e05683ce453e in term 1.
I20260810 18:48:46.458318 27086 raft_consensus.cc:2454] T c573413479ce429a8a03150ca410db1e P 7023e9dd2aaa481cbe00bfe83efac6ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aca3a4b0c4f8484c9a62e05683ce453e in term 1.
I20260810 18:48:46.458607 27320 leader_election.cc:302] T c573413479ce429a8a03150ca410db1e P aca3a4b0c4f8484c9a62e05683ce453e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7023e9dd2aaa481cbe00bfe83efac6ed, aca3a4b0c4f8484c9a62e05683ce453e; no voters:
I20260810 18:48:46.458796 27557 raft_consensus.cc:2781] T c573413479ce429a8a03150ca410db1e P aca3a4b0c4f8484c9a62e05683ce453e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:46.459064 27557 raft_consensus.cc:686] T c573413479ce429a8a03150ca410db1e P aca3a4b0c4f8484c9a62e05683ce453e [term 1 LEADER]: Becoming Leader. State: Replica: aca3a4b0c4f8484c9a62e05683ce453e, State: Running, Role: LEADER
I20260810 18:48:46.459309 27557 consensus_queue.cc:237] T c573413479ce429a8a03150ca410db1e P aca3a4b0c4f8484c9a62e05683ce453e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aca3a4b0c4f8484c9a62e05683ce453e" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45911 } } peers { permanent_uuid: "7023e9dd2aaa481cbe00bfe83efac6ed" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 42891 } } peers { permanent_uuid: "59c9f2d08b94401989fdc32d65539feb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 41981 } }
I20260810 18:48:46.459703 27566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:46.459463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43053 (local address 127.18.168.131:53115)
0810 18:48:46.459571 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:46.459628 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:43053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260810 18:48:46.461714 26933 catalog_manager.cc:5495] T c573413479ce429a8a03150ca410db1e P aca3a4b0c4f8484c9a62e05683ce453e reported cstate change: term changed from 0 to 1, leader changed from <none> to aca3a4b0c4f8484c9a62e05683ce453e (127.18.168.131). New cstate: current_term: 1 leader_uuid: "aca3a4b0c4f8484c9a62e05683ce453e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aca3a4b0c4f8484c9a62e05683ce453e" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7023e9dd2aaa481cbe00bfe83efac6ed" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 42891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59c9f2d08b94401989fdc32d65539feb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 41981 } health_report { overall_health: UNKNOWN } } }
I20260810 18:48:46.475574 27546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:46.472003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60162 (local address 127.18.168.131:45911)
0810 18:48:46.472103 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:46.472115 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:46.472164 (+ 49us) server_negotiation.cc:407] Connection header received
0810 18:48:46.472339 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:46.472348 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:46.472398 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:46.472507 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:48:46.472796 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:46.473293 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:46.474163 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:46.474385 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:46.474497 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:46.474787 (+ 290us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:48:46.475211 (+ 424us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:48:46.475339 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:46.475399 (+ 60us) server_negotiation.cc:299] Negotiation successful
0810 18:48:46.475448 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 18:48:46.478145 27086 raft_consensus.cc:1261] T c573413479ce429a8a03150ca410db1e P 7023e9dd2aaa481cbe00bfe83efac6ed [term 1 FOLLOWER]: Refusing update from remote peer aca3a4b0c4f8484c9a62e05683ce453e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:46.478431 27239 raft_consensus.cc:1261] T c573413479ce429a8a03150ca410db1e P 59c9f2d08b94401989fdc32d65539feb [term 1 FOLLOWER]: Refusing update from remote peer aca3a4b0c4f8484c9a62e05683ce453e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:46.478672 27557 consensus_queue.cc:1035] T c573413479ce429a8a03150ca410db1e P aca3a4b0c4f8484c9a62e05683ce453e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7023e9dd2aaa481cbe00bfe83efac6ed" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 42891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:46.479007 27567 consensus_queue.cc:1035] T c573413479ce429a8a03150ca410db1e P aca3a4b0c4f8484c9a62e05683ce453e [LEADER]: Connected to new peer: Peer: permanent_uuid: "59c9f2d08b94401989fdc32d65539feb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 41981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:46.479710 27138 heartbeater.cc:503] Master 127.18.168.188:46047 was elected leader, sending a full tablet report...
I20260810 18:48:46.481092 27562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:46.480813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43053 (local address 127.18.168.129:45577)
0810 18:48:46.480973 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:46.481032 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:43053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260810 18:48:46.481117 27565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:46.480811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43053 (local address 127.18.168.130:58843)
0810 18:48:46.480988 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:46.481047 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:43053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260810 18:48:46.483680 27566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:46.483416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43053 (local address 127.18.168.131:37667)
0810 18:48:46.483531 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:46.483613 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:43053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 18:48:46.485647 27562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:46.485381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43053 (local address 127.18.168.129:40263)
0810 18:48:46.485517 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:46.485582 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:43053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260810 18:48:46.487804 27566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:46.487544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43053 (local address 127.18.168.131:59661)
0810 18:48:46.487680 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:46.487734 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:43053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260810 18:48:46.488829 27580 mvcc.cc:205] Tried to move back new op lower bound from 7317044127649665024 to 7317044127580278784. Current Snapshot: MvccSnapshot[applied={T|T < 7317044127649665024}]
I20260810 18:48:46.489789 27565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:46.489515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43053 (local address 127.18.168.130:46015)
0810 18:48:46.489674 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:46.489725 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:43053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260810 18:48:46.491019 26932 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47886:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:46.491180 26932 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47886:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:46.494678 26932 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c8aece922db748f48528a46f14ec0d88: Sending DeleteTablet for 3 replicas of tablet c573413479ce429a8a03150ca410db1e
I20260810 18:48:46.495211 27365 tablet_service.cc:1514] Processing DeleteTablet for tablet c573413479ce429a8a03150ca410db1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:46 UTC) from {username='oryx', principal='oryx/127.18.168.188@KRBTEST.COM'} at 127.0.0.1:60146
I20260810 18:48:46.495694 27588 tablet_replica.cc:330] T c573413479ce429a8a03150ca410db1e P aca3a4b0c4f8484c9a62e05683ce453e: stopping tablet replica
I20260810 18:48:46.495837 27588 raft_consensus.cc:2229] T c573413479ce429a8a03150ca410db1e P aca3a4b0c4f8484c9a62e05683ce453e [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:48:46.495993 27062 tablet_service.cc:1514] Processing DeleteTablet for tablet c573413479ce429a8a03150ca410db1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:46 UTC) from {username='oryx', principal='oryx/127.18.168.188@KRBTEST.COM'} at 127.0.0.1:41670
I20260810 18:48:46.496006 27215 tablet_service.cc:1514] Processing DeleteTablet for tablet c573413479ce429a8a03150ca410db1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:46 UTC) from {username='oryx', principal='oryx/127.18.168.188@KRBTEST.COM'} at 127.0.0.1:58808
I20260810 18:48:46.496111 27588 raft_consensus.cc:2258] T c573413479ce429a8a03150ca410db1e P aca3a4b0c4f8484c9a62e05683ce453e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:46.496443 27589 tablet_replica.cc:330] T c573413479ce429a8a03150ca410db1e P 7023e9dd2aaa481cbe00bfe83efac6ed: stopping tablet replica
I20260810 18:48:46.496634 27589 raft_consensus.cc:2229] T c573413479ce429a8a03150ca410db1e P 7023e9dd2aaa481cbe00bfe83efac6ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:46.496868 27588 ts_tablet_manager.cc:1905] T c573413479ce429a8a03150ca410db1e P aca3a4b0c4f8484c9a62e05683ce453e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:46.496881 27589 raft_consensus.cc:2258] T c573413479ce429a8a03150ca410db1e P 7023e9dd2aaa481cbe00bfe83efac6ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:46.497290 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 26890
I20260810 18:48:46.502722 27588 ts_tablet_manager.cc:1918] T c573413479ce429a8a03150ca410db1e P aca3a4b0c4f8484c9a62e05683ce453e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:48:46.502833 27588 log.cc:1196] T c573413479ce429a8a03150ca410db1e P aca3a4b0c4f8484c9a62e05683ce453e: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-2/wal/wals/c573413479ce429a8a03150ca410db1e
I20260810 18:48:46.503223 27588 ts_tablet_manager.cc:1939] T c573413479ce429a8a03150ca410db1e P aca3a4b0c4f8484c9a62e05683ce453e: Deleting consensus metadata
I20260810 18:48:46.504205 26918 catalog_manager.cc:4841] TS aca3a4b0c4f8484c9a62e05683ce453e (127.18.168.131:45911): tablet c573413479ce429a8a03150ca410db1e (table test-table [id=5fbe1d74cc154db5ad9c7fb9b198b32f]) successfully deleted
I20260810 18:48:46.506347 27590 tablet_replica.cc:330] T c573413479ce429a8a03150ca410db1e P 59c9f2d08b94401989fdc32d65539feb: stopping tablet replica
I20260810 18:48:46.506503 27590 raft_consensus.cc:2229] T c573413479ce429a8a03150ca410db1e P 59c9f2d08b94401989fdc32d65539feb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:46.506664 27590 raft_consensus.cc:2258] T c573413479ce429a8a03150ca410db1e P 59c9f2d08b94401989fdc32d65539feb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:46.507339 27590 ts_tablet_manager.cc:1905] T c573413479ce429a8a03150ca410db1e P 59c9f2d08b94401989fdc32d65539feb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:46.509599 27590 ts_tablet_manager.cc:1918] T c573413479ce429a8a03150ca410db1e P 59c9f2d08b94401989fdc32d65539feb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:48:46.509703 27590 log.cc:1196] T c573413479ce429a8a03150ca410db1e P 59c9f2d08b94401989fdc32d65539feb: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387687857264-19106-0/minicluster-data/ts-1/wal/wals/c573413479ce429a8a03150ca410db1e
I20260810 18:48:46.510025 27590 ts_tablet_manager.cc:1939] T c573413479ce429a8a03150ca410db1e P 59c9f2d08b94401989fdc32d65539feb: Deleting consensus metadata
I20260810 18:48:46.510735 26919 catalog_manager.cc:4841] TS 59c9f2d08b94401989fdc32d65539feb (127.18.168.130:41981): tablet c573413479ce429a8a03150ca410db1e (table test-table [id=5fbe1d74cc154db5ad9c7fb9b198b32f]) successfully deleted
I20260810 18:48:46.514863 27549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:46.514565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:42891 (local address 127.0.0.1:41682)
0810 18:48:46.514689 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:46.514749 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.129:42891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260810 18:48:46.515187 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 27146
W20260810 18:48:46.515172 26917 catalog_manager.cc:4584] TS 7023e9dd2aaa481cbe00bfe83efac6ed (127.18.168.129:42891): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c573413479ce429a8a03150ca410db1e: Network error: Client connection negotiation failed: client connection to 127.18.168.129:42891: connect: Connection refused (error 111)
I20260810 18:48:46.525637 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 27300
I20260810 18:48:46.533402 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 26569
I20260810 18:48:46.541435 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 26687
2026-08-10T18:48:46Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5569 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:48:46 dist-test-slave-cbsf krb5kdc[27593](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:48:46 dist-test-slave-cbsf krb5kdc[27593](info): set up 2 sockets
Aug 10 18:48:46 dist-test-slave-cbsf krb5kdc[27593](info): commencing operation
krb5kdc: starting...
W20260810 18:48:47.713267 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.146s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:48:47 dist-test-slave-cbsf krb5kdc[27593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:48:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:48:47Z Disabled control of system clock
WARNING: no policy specified for oryx/127.18.168.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.168.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.168.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.190.keytab.
Entry for principal oryx/127.18.168.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.190.keytab.
I20260810 18:48:47.841650 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:43829
--webserver_interface=127.18.168.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.190.keytab
--principal=oryx/127.18.168.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:41195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:43829
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:47.952232 27607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:47.952489 27607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:47.952528 27607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:47.956481 27607 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:48:47.956539 27607 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 18:48:47.956555 27607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:47.956573 27607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:47.956588 27607 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:48:47.956602 27607 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:48:47.961690 27607 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:41195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:43829
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.190.keytab
--principal=oryx/127.18.168.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:43829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:47.962719 27607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:47.963698 27607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:47.972358 27616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:47.972573 27607 server_base.cc:1034] running on GCE node
W20260810 18:48:47.972386 27613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:47.972332 27614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:47.973333 27607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:47.974442 27607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:47.975664 27607 hybrid_clock.cc:648] HybridClock initialized: now 1786387727975632 us; error 50 us; skew 500 ppm
Aug 10 18:48:47 dist-test-slave-cbsf krb5kdc[27593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387727, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.168.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:47.978443 27607 init.cc:377] Logged in from keytab as oryx/127.18.168.190@KRBTEST.COM (short username oryx)
I20260810 18:48:47.979374 27607 webserver.cc:460] Webserver started at http://127.18.168.190:39123/ using document root <none> and password file <none>
I20260810 18:48:47.979960 27607 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:47.980054 27607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:47.980296 27607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:47.982445 27607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "dfbc86f800924b13a83b5a0e342be613"
format_stamp: "Formatted at 2026-08-10 18:48:47 on dist-test-slave-cbsf"
server_key: "270d32e02d181fc34d447c447b8f9c1d"
server_key_iv: "e3405ba360588a6c36d9542f1e5a10fb"
server_key_version: "encryptionkey@0"
I20260810 18:48:47.983042 27607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "dfbc86f800924b13a83b5a0e342be613"
format_stamp: "Formatted at 2026-08-10 18:48:47 on dist-test-slave-cbsf"
server_key: "270d32e02d181fc34d447c447b8f9c1d"
server_key_iv: "e3405ba360588a6c36d9542f1e5a10fb"
server_key_version: "encryptionkey@0"
I20260810 18:48:47.986929 27607 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260810 18:48:47.989359 27623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:47.990518 27607 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 18:48:47.990680 27607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "dfbc86f800924b13a83b5a0e342be613"
format_stamp: "Formatted at 2026-08-10 18:48:47 on dist-test-slave-cbsf"
server_key: "270d32e02d181fc34d447c447b8f9c1d"
server_key_iv: "e3405ba360588a6c36d9542f1e5a10fb"
server_key_version: "encryptionkey@0"
I20260810 18:48:47.990813 27607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:48.001125 27607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:48.003808 27607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:48.004037 27607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:48.012177 27607 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:43829
I20260810 18:48:48.012192 27694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:43829 every 8 connection(s)
I20260810 18:48:48.013134 27607 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:48:48.015841 27695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:48.017932 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 27607
I20260810 18:48:48.018083 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:48:48.018390 19106 external_mini_cluster.cc:1428] Setting key 0d2718ca073235e9676e566e51a5b637
I20260810 18:48:48.021916 27695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfbc86f800924b13a83b5a0e342be613: Bootstrap starting.
I20260810 18:48:48.024437 27695 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dfbc86f800924b13a83b5a0e342be613: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:48.025345 27695 log.cc:824] T 00000000000000000000000000000000 P dfbc86f800924b13a83b5a0e342be613: Log is configured to *not* fsync() on all Append() calls
Aug 10 18:48:48 dist-test-slave-cbsf krb5kdc[27593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM
I20260810 18:48:48.027525 27695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfbc86f800924b13a83b5a0e342be613: No bootstrap required, opened a new log
I20260810 18:48:48.031033 27695 raft_consensus.cc:348] T 00000000000000000000000000000000 P dfbc86f800924b13a83b5a0e342be613 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfbc86f800924b13a83b5a0e342be613" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43829 } }
I20260810 18:48:48.031280 27695 raft_consensus.cc:374] T 00000000000000000000000000000000 P dfbc86f800924b13a83b5a0e342be613 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:48.031385 27695 raft_consensus.cc:729] T 00000000000000000000000000000000 P dfbc86f800924b13a83b5a0e342be613 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfbc86f800924b13a83b5a0e342be613, State: Initialized, Role: FOLLOWER
I20260810 18:48:48.031924 27695 consensus_queue.cc:260] T 00000000000000000000000000000000 P dfbc86f800924b13a83b5a0e342be613 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfbc86f800924b13a83b5a0e342be613" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43829 } }
I20260810 18:48:48.032083 27695 raft_consensus.cc:388] T 00000000000000000000000000000000 P dfbc86f800924b13a83b5a0e342be613 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:48:48.032145 27695 raft_consensus.cc:482] T 00000000000000000000000000000000 P dfbc86f800924b13a83b5a0e342be613 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:48:48.032238 27695 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dfbc86f800924b13a83b5a0e342be613 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:48.032964 27700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:48.019791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52546 (local address 127.18.168.190:43829)
0810 18:48:48.020179 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:48.020203 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:48.020238 (+ 35us) server_negotiation.cc:407] Connection header received
0810 18:48:48.020887 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:48.020904 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:48.021213 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:48.021483 (+ 270us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:48.022421 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.023396 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.024572 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.025096 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.025215 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:48.028093 (+ 2878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:48.028098 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:48.028110 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:48.028136 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:48.029879 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:48.030338 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:48.030344 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:48.030348 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:48.030456 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:48.030749 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:48.030752 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:48.030754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:48.031121 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:48.031330 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:48.031685 (+ 355us) server_negotiation.cc:299] Negotiation successful
0810 18:48:48.031976 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":129,"threads_started":1}
I20260810 18:48:48.033291 27695 raft_consensus.cc:504] T 00000000000000000000000000000000 P dfbc86f800924b13a83b5a0e342be613 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfbc86f800924b13a83b5a0e342be613" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43829 } }
I20260810 18:48:48.033633 27695 leader_election.cc:302] T 00000000000000000000000000000000 P dfbc86f800924b13a83b5a0e342be613 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dfbc86f800924b13a83b5a0e342be613; no voters:
I20260810 18:48:48.033981 27695 leader_election.cc:288] T 00000000000000000000000000000000 P dfbc86f800924b13a83b5a0e342be613 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:48:48.034073 27702 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dfbc86f800924b13a83b5a0e342be613 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:48.034303 27702 raft_consensus.cc:686] T 00000000000000000000000000000000 P dfbc86f800924b13a83b5a0e342be613 [term 1 LEADER]: Becoming Leader. State: Replica: dfbc86f800924b13a83b5a0e342be613, State: Running, Role: LEADER
I20260810 18:48:48.034653 27702 consensus_queue.cc:237] T 00000000000000000000000000000000 P dfbc86f800924b13a83b5a0e342be613 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfbc86f800924b13a83b5a0e342be613" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43829 } }
I20260810 18:48:48.035324 27695 sys_catalog.cc:564] T 00000000000000000000000000000000 P dfbc86f800924b13a83b5a0e342be613 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:48:48.035945 27704 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfbc86f800924b13a83b5a0e342be613 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dfbc86f800924b13a83b5a0e342be613. Latest consensus state: current_term: 1 leader_uuid: "dfbc86f800924b13a83b5a0e342be613" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfbc86f800924b13a83b5a0e342be613" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43829 } } }
I20260810 18:48:48.036159 27704 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfbc86f800924b13a83b5a0e342be613 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:48.036435 27703 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfbc86f800924b13a83b5a0e342be613 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dfbc86f800924b13a83b5a0e342be613" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfbc86f800924b13a83b5a0e342be613" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43829 } } }
I20260810 18:48:48.036532 27703 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfbc86f800924b13a83b5a0e342be613 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:48.036602 27710 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:48:48.039819 27710 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:48:48.045177 27710 catalog_manager.cc:1347] Generated new cluster ID: 154fb2076c4d4fafa1e2d362ab5e42c1
I20260810 18:48:48.045248 27710 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:48:48.073801 27710 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:48:48.075156 27710 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:48:48.083158 27710 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dfbc86f800924b13a83b5a0e342be613: Generated new TSK 0
I20260810 18:48:48.083927 27710 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.18.168.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.168.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.168.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.129.keytab.
Entry for principal oryx/127.18.168.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.129.keytab.
I20260810 18:48:48.116184 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.129:0
--local_ip_for_outbound_sockets=127.18.168.129
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:43829
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.129.keytab
--principal=oryx/127.18.168.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:41195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:48.236344 27723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:48.236657 27723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:48.236776 27723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:48.240819 27723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:48.240868 27723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:48.240902 27723 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 18:48:48.240931 27723 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.129
I20260810 18:48:48.245884 27723 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:41195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.129.keytab
--principal=oryx/127.18.168.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.168.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.168.190:43829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.129
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:48.246799 27723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:48.247756 27723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:48.257186 27728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:48.257340 27723 server_base.cc:1034] running on GCE node
W20260810 18:48:48.257184 27729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:48.257221 27732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:48.257893 27723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:48.258546 27723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:48.259706 27723 hybrid_clock.cc:648] HybridClock initialized: now 1786387728259693 us; error 23 us; skew 500 ppm
Aug 10 18:48:48 dist-test-slave-cbsf krb5kdc[27593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387728, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.168.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:48.262514 27723 init.cc:377] Logged in from keytab as oryx/127.18.168.129@KRBTEST.COM (short username oryx)
I20260810 18:48:48.263339 27723 webserver.cc:460] Webserver started at http://127.18.168.129:44299/ using document root <none> and password file <none>
I20260810 18:48:48.263921 27723 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:48.263981 27723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:48.264242 27723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:48.266230 27723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-0/data/instance:
uuid: "87492194659641e2a9141f7233bcc623"
format_stamp: "Formatted at 2026-08-10 18:48:48 on dist-test-slave-cbsf"
server_key: "606d24faccb793382f4a967fc4a50831"
server_key_iv: "2bd51eb3c625d7ee5ef77f666795e79f"
server_key_version: "encryptionkey@0"
I20260810 18:48:48.266773 27723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance:
uuid: "87492194659641e2a9141f7233bcc623"
format_stamp: "Formatted at 2026-08-10 18:48:48 on dist-test-slave-cbsf"
server_key: "606d24faccb793382f4a967fc4a50831"
server_key_iv: "2bd51eb3c625d7ee5ef77f666795e79f"
server_key_version: "encryptionkey@0"
I20260810 18:48:48.270380 27723 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260810 18:48:48.272876 27739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:48.274173 27723 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:48:48.274314 27723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-0/wal
uuid: "87492194659641e2a9141f7233bcc623"
format_stamp: "Formatted at 2026-08-10 18:48:48 on dist-test-slave-cbsf"
server_key: "606d24faccb793382f4a967fc4a50831"
server_key_iv: "2bd51eb3c625d7ee5ef77f666795e79f"
server_key_version: "encryptionkey@0"
I20260810 18:48:48.274425 27723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:48.292408 27723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:48.295709 27723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:48.295949 27723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:48.297679 27723 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:48.297739 27723 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:48.297821 27723 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:48.297874 27723 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 18:48:48 dist-test-slave-cbsf krb5kdc[27593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387728, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.168.129@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM
I20260810 18:48:48.311218 27723 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.129:43315
I20260810 18:48:48.311322 27875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.129:43315 every 8 connection(s)
I20260810 18:48:48.312217 27723 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
I20260810 18:48:48.312429 27700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:48.298445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:47737 (local address 127.18.168.190:43829)
0810 18:48:48.298621 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:48.298629 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:48.299457 (+ 828us) server_negotiation.cc:407] Connection header received
0810 18:48:48.300485 (+ 1028us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:48.300493 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:48.300564 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:48.300715 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:48.302091 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.303061 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.304342 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.304584 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.304727 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:48.308096 (+ 3369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:48.308099 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:48.308101 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:48.308125 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:48.310176 (+ 2051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:48.310694 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:48.310698 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:48.310700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:48.310821 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:48.311245 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:48.311249 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:48.311251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:48.311509 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:48.311628 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:48.312125 (+ 497us) server_negotiation.cc:299] Negotiation successful
0810 18:48:48.312254 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 18:48:48.313385 27751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:48.298827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43829 (local address 127.18.168.129:47737)
0810 18:48:48.299283 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:48.299334 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:48.300225 (+ 891us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:48.300919 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:48.300930 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:48.301414 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:48.301943 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:48.301956 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.303226 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.303230 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:48.304210 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:48.304221 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.304779 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.304783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:48.304961 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:48.305527 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:48.305546 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:48.307911 (+ 2365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:48.310304 (+ 2393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:48.310309 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:48.310320 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:48.310571 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:48.310955 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:48.310959 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:48.310961 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:48.311139 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:48.311627 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:48.311634 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:48.311882 (+ 248us) client_negotiation.cc:769] Sending connection context
0810 18:48:48.312065 (+ 183us) client_negotiation.cc:240] Negotiation successful
0810 18:48:48.312388 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":126,"threads_started":1}
I20260810 18:48:48.313923 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 27723
I20260810 18:48:48.314078 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance
I20260810 18:48:48.314476 19106 external_mini_cluster.cc:1428] Setting key 4a470ed0e69db9120560bc55ee8f221b
I20260810 18:48:48.314808 27876 heartbeater.cc:347] Connected to a master server at 127.18.168.190:43829
I20260810 18:48:48.315039 27876 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:48.315795 27876 heartbeater.cc:511] Master 127.18.168.190:43829 requested a full tablet report, sending...
I20260810 18:48:48.317571 27642 ts_manager.cc:194] Registered new tserver with Master: 87492194659641e2a9141f7233bcc623 (127.18.168.129:43315)
I20260810 18:48:48.318679 27642 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.168.129@KRBTEST.COM'} at 127.18.168.129:47737
I20260810 18:48:48.326434 27882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:48.317350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43829 (local address 127.18.168.129:49793)
0810 18:48:48.319371 (+ 2021us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:48.319384 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:48.319486 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:48.319710 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:48.319713 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:48.319906 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:48.319993 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:48.319998 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.321041 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.321043 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:48.321584 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:48.321592 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.322210 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.322213 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:48.322281 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:48.322785 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:48.322797 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:48.323100 (+ 303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:48.325029 (+ 1929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:48.325032 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:48.325033 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:48.325220 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:48.325519 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:48.325521 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:48.325522 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:48.325600 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:48.326040 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:48.326042 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:48.326116 (+ 74us) client_negotiation.cc:769] Sending connection context
0810 18:48:48.326198 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 18:48:48.326296 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1922,"thread_start_us":94,"threads_started":1}
I20260810 18:48:48.326619 27700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:48.317372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:49793 (local address 127.18.168.190:43829)
0810 18:48:48.317509 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:48.317515 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:48.319448 (+ 1933us) server_negotiation.cc:407] Connection header received
0810 18:48:48.319559 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:48.319562 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:48.319606 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:48.319696 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:48.320109 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.320896 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.321717 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.322093 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.322218 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:48.323249 (+ 1031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:48.323255 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:48.323261 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:48.323294 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:48.324925 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:48.325322 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:48.325328 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:48.325329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:48.325419 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:48.325699 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:48.325705 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:48.325708 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:48.325938 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:48.326053 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:48.326296 (+ 243us) server_negotiation.cc:299] Negotiation successful
0810 18:48:48.326428 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
WARNING: no policy specified for oryx/127.18.168.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.168.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.168.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.130.keytab.
Entry for principal oryx/127.18.168.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.130.keytab.
I20260810 18:48:48.345913 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.130:0
--local_ip_for_outbound_sockets=127.18.168.130
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:43829
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.130.keytab
--principal=oryx/127.18.168.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:41195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:48.456573 27885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:48.456894 27885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:48.456979 27885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:48.460922 27885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:48.460989 27885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:48.461052 27885 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 18:48:48.461138 27885 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.130
I20260810 18:48:48.466137 27885 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:41195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.130.keytab
--principal=oryx/127.18.168.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.168.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.168.190:43829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.130
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:48.467233 27885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:48.468225 27885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:48.477942 27891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:48.478121 27885 server_base.cc:1034] running on GCE node
W20260810 18:48:48.477953 27892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:48.477952 27894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:48.478581 27885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:48.479242 27885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:48.480400 27885 hybrid_clock.cc:648] HybridClock initialized: now 1786387728480383 us; error 28 us; skew 500 ppm
Aug 10 18:48:48 dist-test-slave-cbsf krb5kdc[27593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387728, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.168.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:48.483201 27885 init.cc:377] Logged in from keytab as oryx/127.18.168.130@KRBTEST.COM (short username oryx)
I20260810 18:48:48.484067 27885 webserver.cc:460] Webserver started at http://127.18.168.130:43541/ using document root <none> and password file <none>
I20260810 18:48:48.484629 27885 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:48.484740 27885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:48.484987 27885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:48.486937 27885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-1/data/instance:
uuid: "7895952636a34ed7b8d53efba58d8dbb"
format_stamp: "Formatted at 2026-08-10 18:48:48 on dist-test-slave-cbsf"
server_key: "8d9394430b4390660e91766045125a7a"
server_key_iv: "6f862140deeb544906a8d5da96475633"
server_key_version: "encryptionkey@0"
I20260810 18:48:48.487540 27885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance:
uuid: "7895952636a34ed7b8d53efba58d8dbb"
format_stamp: "Formatted at 2026-08-10 18:48:48 on dist-test-slave-cbsf"
server_key: "8d9394430b4390660e91766045125a7a"
server_key_iv: "6f862140deeb544906a8d5da96475633"
server_key_version: "encryptionkey@0"
I20260810 18:48:48.491283 27885 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 18:48:48.493769 27902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:48.495049 27885 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:48:48.495206 27885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-1/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-1/wal
uuid: "7895952636a34ed7b8d53efba58d8dbb"
format_stamp: "Formatted at 2026-08-10 18:48:48 on dist-test-slave-cbsf"
server_key: "8d9394430b4390660e91766045125a7a"
server_key_iv: "6f862140deeb544906a8d5da96475633"
server_key_version: "encryptionkey@0"
I20260810 18:48:48.495314 27885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:48.521574 27885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:48.524634 27885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:48.524886 27885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:48.526726 27885 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:48.526798 27885 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:48.526859 27885 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:48.526895 27885 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 18:48:48 dist-test-slave-cbsf krb5kdc[27593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387728, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.168.130@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM
I20260810 18:48:48.539960 27885 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.130:37203
I20260810 18:48:48.540036 28043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.130:37203 every 8 connection(s)
I20260810 18:48:48.540943 27885 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
I20260810 18:48:48.542145 27700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:48.527254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:53715 (local address 127.18.168.190:43829)
0810 18:48:48.527390 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:48.527393 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:48.528415 (+ 1022us) server_negotiation.cc:407] Connection header received
0810 18:48:48.529507 (+ 1092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:48.529512 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:48.529574 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:48.529687 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:48.530971 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.531509 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.533015 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.533246 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.533362 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:48.537460 (+ 4098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:48.537466 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:48.537470 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:48.537506 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:48.539382 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:48.540012 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:48.540016 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:48.540018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:48.540119 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:48.540548 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:48.540558 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:48.540559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:48.540822 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:48.540937 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:48.541918 (+ 981us) server_negotiation.cc:299] Negotiation successful
0810 18:48:48.542023 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 18:48:48.542614 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 27885
I20260810 18:48:48.542733 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance
I20260810 18:48:48.543037 19106 external_mini_cluster.cc:1428] Setting key a7b9be692169ba4c24bb5c4a6f387050
I20260810 18:48:48.543296 27913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:48.527744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43829 (local address 127.18.168.130:53715)
0810 18:48:48.528275 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:48.528315 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:48.529182 (+ 867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:48.529881 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:48.529890 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:48.530322 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:48.530788 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:48.530820 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.531658 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.531661 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:48.532887 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:48.532898 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.533406 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.533410 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:48.533629 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:48.534259 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:48.534289 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:48.537189 (+ 2900us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:48.539529 (+ 2340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:48.539535 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:48.539549 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:48.539859 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:48.540239 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:48.540243 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:48.540246 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:48.540436 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:48.540935 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:48.540942 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:48.541236 (+ 294us) client_negotiation.cc:769] Sending connection context
0810 18:48:48.541451 (+ 215us) client_negotiation.cc:240] Negotiation successful
0810 18:48:48.542429 (+ 978us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":158,"threads_started":1}
I20260810 18:48:48.544754 28044 heartbeater.cc:347] Connected to a master server at 127.18.168.190:43829
I20260810 18:48:48.544910 28044 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:48.545516 28044 heartbeater.cc:511] Master 127.18.168.190:43829 requested a full tablet report, sending...
I20260810 18:48:48.546672 27642 ts_manager.cc:194] Registered new tserver with Master: 7895952636a34ed7b8d53efba58d8dbb (127.18.168.130:37203)
I20260810 18:48:48.547325 27642 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.168.130@KRBTEST.COM'} at 127.18.168.130:53715
WARNING: no policy specified for oryx/127.18.168.131@KRBTEST.COM; defaulting to no policy
I20260810 18:48:48.555486 28051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:48.547252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43829 (local address 127.18.168.130:49567)
0810 18:48:48.547579 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:48.547597 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:48.547752 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:48.548038 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:48.548043 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:48.548291 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:48.548435 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:48.548444 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.549474 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.549480 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:48.550548 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:48.550561 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.551010 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.551017 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:48.551142 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:48.551654 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:48.551674 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:48.552164 (+ 490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:48.553869 (+ 1705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:48.553873 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:48.553876 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:48.554141 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:48.554441 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:48.554444 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:48.554446 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:48.554538 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:48.555072 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:48.555077 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:48.555174 (+ 97us) client_negotiation.cc:769] Sending connection context
0810 18:48:48.555264 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 18:48:48.555362 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":82,"threads_started":1}
I20260810 18:48:48.555554 27700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:48.547321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:49567 (local address 127.18.168.190:43829)
0810 18:48:48.547447 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:48.547451 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:48.547677 (+ 226us) server_negotiation.cc:407] Connection header received
0810 18:48:48.547864 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:48.547869 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:48.547915 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:48.548001 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:48.548582 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.549324 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.550666 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.550874 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.550961 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:48.552306 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:48.552308 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:48.552310 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:48.552331 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:48.553753 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:48.554272 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:48.554274 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:48.554275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:48.554344 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:48.554659 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:48.554663 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:48.554664 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:48.554960 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:48.555052 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:48.555325 (+ 273us) server_negotiation.cc:299] Negotiation successful
0810 18:48:48.555415 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.168.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.168.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.131.keytab.
Entry for principal oryx/127.18.168.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.131.keytab.
I20260810 18:48:48.573733 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.131:0
--local_ip_for_outbound_sockets=127.18.168.131
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:43829
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.131.keytab
--principal=oryx/127.18.168.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:41195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:48.685459 28053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:48.685691 28053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:48.685734 28053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:48.689638 28053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:48.689697 28053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:48.689735 28053 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 18:48:48.689767 28053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.131
I20260810 18:48:48.694886 28053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:41195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.131.keytab
--principal=oryx/127.18.168.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.168.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.168.190:43829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.131
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:48.695823 28053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:48.696811 28053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:48.706296 28059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:48.706357 28053 server_base.cc:1034] running on GCE node
W20260810 18:48:48.706264 28062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:48.706271 28060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:48.706874 28053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:48.707492 28053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:48.708657 28053 hybrid_clock.cc:648] HybridClock initialized: now 1786387728708633 us; error 28 us; skew 500 ppm
Aug 10 18:48:48 dist-test-slave-cbsf krb5kdc[27593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387728, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.168.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:48.711367 28053 init.cc:377] Logged in from keytab as oryx/127.18.168.131@KRBTEST.COM (short username oryx)
I20260810 18:48:48.712190 28053 webserver.cc:460] Webserver started at http://127.18.168.131:33029/ using document root <none> and password file <none>
I20260810 18:48:48.712778 28053 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:48.712845 28053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:48.713016 28053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:48.715103 28053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-2/data/instance:
uuid: "a07e4e0445a140de92c89b90c930e418"
format_stamp: "Formatted at 2026-08-10 18:48:48 on dist-test-slave-cbsf"
server_key: "26a596b0f18471694012d215fc1d2628"
server_key_iv: "992fe98230211f3476c55085655c0b2b"
server_key_version: "encryptionkey@0"
I20260810 18:48:48.715701 28053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance:
uuid: "a07e4e0445a140de92c89b90c930e418"
format_stamp: "Formatted at 2026-08-10 18:48:48 on dist-test-slave-cbsf"
server_key: "26a596b0f18471694012d215fc1d2628"
server_key_iv: "992fe98230211f3476c55085655c0b2b"
server_key_version: "encryptionkey@0"
I20260810 18:48:48.719362 28053 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260810 18:48:48.721722 28069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:48.722865 28053 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:48:48.722986 28053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-2/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-2/wal
uuid: "a07e4e0445a140de92c89b90c930e418"
format_stamp: "Formatted at 2026-08-10 18:48:48 on dist-test-slave-cbsf"
server_key: "26a596b0f18471694012d215fc1d2628"
server_key_iv: "992fe98230211f3476c55085655c0b2b"
server_key_version: "encryptionkey@0"
I20260810 18:48:48.723089 28053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:48.738853 28053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:48.741613 28053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:48.741809 28053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:48.743281 28053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:48.743356 28053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:48.743427 28053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:48.743479 28053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 18:48:48 dist-test-slave-cbsf krb5kdc[27593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387728, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.168.131@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM
I20260810 18:48:48.754709 28053 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.131:43327
I20260810 18:48:48.754763 28216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.131:43327 every 8 connection(s)
I20260810 18:48:48.755586 28053 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
I20260810 18:48:48.757786 27700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:48.744093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:47517 (local address 127.18.168.190:43829)
0810 18:48:48.744234 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:48.744237 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:48.745236 (+ 999us) server_negotiation.cc:407] Connection header received
0810 18:48:48.746245 (+ 1009us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:48.746248 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:48.746307 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:48.746409 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:48.747809 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.748724 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.749912 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.750229 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.750366 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:48.754062 (+ 3696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:48.754065 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:48.754067 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:48.754092 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:48.755676 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:48.756363 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:48.756368 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:48.756371 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:48.756468 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:48.756845 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:48.756849 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:48.756850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:48.757022 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:48.757103 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:48.757567 (+ 464us) server_negotiation.cc:299] Negotiation successful
0810 18:48:48.757668 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 18:48:48.758544 28081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:48.744607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43829 (local address 127.18.168.131:47517)
0810 18:48:48.745093 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:48.745132 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:48.746040 (+ 908us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:48.746713 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:48.746723 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:48.747139 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:48.747644 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:48.747659 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.748863 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.748867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:48.749729 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:48.749739 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.750411 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.750415 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:48.750630 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:48.751180 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:48.751205 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:48.753873 (+ 2668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:48.755808 (+ 1935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:48.755814 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:48.755828 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:48.756201 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:48.756569 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:48.756573 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:48.756575 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:48.756761 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:48.757113 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:48.757118 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:48.757357 (+ 239us) client_negotiation.cc:769] Sending connection context
0810 18:48:48.757501 (+ 144us) client_negotiation.cc:240] Negotiation successful
0810 18:48:48.757729 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":120,"threads_started":1}
I20260810 18:48:48.759735 28217 heartbeater.cc:347] Connected to a master server at 127.18.168.190:43829
I20260810 18:48:48.759858 28217 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:48.760308 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 28053
I20260810 18:48:48.760385 28217 heartbeater.cc:511] Master 127.18.168.190:43829 requested a full tablet report, sending...
I20260810 18:48:48.760437 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance
I20260810 18:48:48.760784 19106 external_mini_cluster.cc:1428] Setting key 0c8fbc9adbae5b436a38f83fd6370c02
I20260810 18:48:48.761638 27641 ts_manager.cc:194] Registered new tserver with Master: a07e4e0445a140de92c89b90c930e418 (127.18.168.131:43327)
I20260810 18:48:48.762298 27641 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.168.131@KRBTEST.COM'} at 127.18.168.131:47517
I20260810 18:48:48.764432 19106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:48:48.769654 27700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:48.761932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:49341 (local address 127.18.168.190:43829)
0810 18:48:48.762919 (+ 987us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:48.762926 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:48.762941 (+ 15us) server_negotiation.cc:407] Connection header received
0810 18:48:48.762978 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:48.762980 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:48.763021 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:48.763095 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:48.763579 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.764066 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.764997 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.765215 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.765319 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:48.766400 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:48.766407 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:48.766413 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:48.766471 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:48.767917 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:48.768361 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:48.768363 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:48.768364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:48.768431 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:48.768765 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:48.768768 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:48.768770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:48.769010 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:48.769126 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:48.769421 (+ 295us) server_negotiation.cc:299] Negotiation successful
0810 18:48:48.769524 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":909}
I20260810 18:48:48.769673 28226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:48.761870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43829 (local address 127.18.168.131:49341)
0810 18:48:48.762141 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:48.762159 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:48.762225 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:48.763211 (+ 986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:48.763213 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:48.763370 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:48.763461 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:48.763466 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.764201 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.764207 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:48.764868 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:48.764878 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.765323 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.765327 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:48.765421 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:48.765911 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:48.765927 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:48.766229 (+ 302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:48.768029 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:48.768034 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:48.768038 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:48.768275 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:48.768538 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:48.768541 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:48.768543 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:48.768632 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:48.769116 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:48.769119 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:48.769217 (+ 98us) client_negotiation.cc:769] Sending connection context
0810 18:48:48.769343 (+ 126us) client_negotiation.cc:240] Negotiation successful
0810 18:48:48.769478 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":110,"threads_started":1}
Aug 10 18:48:48 dist-test-slave-cbsf krb5kdc[27593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387728, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 18:48:48 dist-test-slave-cbsf krb5kdc[27593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:48:48 dist-test-slave-cbsf krb5kdc[27593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
W20260810 18:48:48.790328 27700 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:48.784339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52548 (local address 127.18.168.190:43829)
0810 18:48:48.784473 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:48.784476 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:48.784487 (+ 11us) server_negotiation.cc:407] Connection header received
0810 18:48:48.784518 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:48.784520 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:48.784556 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:48.784641 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:48.785394 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.785878 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.786777 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.786982 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.787094 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:48.790155 (+ 3061us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52548: BlockingRecv error: recv got EOF from 127.0.0.1:52548 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 10 18:48:48 dist-test-slave-cbsf krb5kdc[27593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387728, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM
I20260810 18:48:48.799633 27700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:48.791851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52562 (local address 127.18.168.190:43829)
0810 18:48:48.791972 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:48.791976 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:48.792111 (+ 135us) server_negotiation.cc:407] Connection header received
0810 18:48:48.792211 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:48.792213 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:48.792247 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:48.792315 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:48.792953 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.793406 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.794261 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.794463 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.794546 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:48.796650 (+ 2104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:48.796652 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:48.796654 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:48.796674 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:48.797954 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:48.798327 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:48.798329 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:48.798330 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:48.798419 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:48.798745 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:48.798749 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:48.798751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:48.798934 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:48.799072 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:48.799335 (+ 263us) server_negotiation.cc:299] Negotiation successful
0810 18:48:48.799445 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 18:48:48.802564 27641 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52562:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:48:48.805579 27641 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:48:48.820000 28242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:48.813925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:37203 (local address 127.0.0.1:39090)
0810 18:48:48.814625 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:48.814675 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:48.814819 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:48.815532 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:48.815538 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:48.815563 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:48.815700 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:48.815707 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.817100 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.817103 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:48.818598 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:48.818610 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.819444 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.819451 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:48.819559 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:48.819596 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 18:48:48.819708 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 18:48:48.819826 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"mutex_wait_us":43,"thread_start_us":61,"threads_started":1}
I20260810 18:48:48.820139 28243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:48.814637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.131:43327 (local address 127.0.0.1:39218)
0810 18:48:48.816113 (+ 1476us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:48.816127 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:48.816214 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:48.816646 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:48.816648 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:48.816668 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:48.816797 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:48.816803 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.817809 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.817814 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:48.818639 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:48.818648 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.819726 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.819729 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:48.819819 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:48.819840 (+ 21us) client_negotiation.cc:769] Sending connection context
0810 18:48:48.819917 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 18:48:48.819971 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1407,"mutex_wait_us":1,"thread_start_us":94,"threads_started":1}
I20260810 18:48:48.820562 28244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:48.814076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39090 (local address 127.18.168.130:37203)
0810 18:48:48.814427 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:48.814440 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:48.815017 (+ 577us) server_negotiation.cc:407] Connection header received
0810 18:48:48.815107 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:48.815112 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:48.815336 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:48.815530 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:48.815818 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.816990 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.818736 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.819306 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.819463 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:48.819606 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:48.820341 (+ 735us) server_negotiation.cc:299] Negotiation successful
0810 18:48:48.820417 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":155,"threads_started":1}
I20260810 18:48:48.820868 28245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:48.814055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:43315 (local address 127.0.0.1:51332)
0810 18:48:48.815394 (+ 1339us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:48.815409 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:48.815500 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:48.815881 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:48.815884 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:48.815902 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:48.816007 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:48.816016 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.817966 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.817969 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:48.819167 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:48.819179 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.820405 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.820408 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:48.820500 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:48.820527 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 18:48:48.820604 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 18:48:48.820725 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1272,"mutex_wait_us":33,"thread_start_us":86,"threads_started":1}
I20260810 18:48:48.821169 28246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:48.814836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39218 (local address 127.18.168.131:43327)
0810 18:48:48.815245 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:48.815255 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:48.816205 (+ 950us) server_negotiation.cc:407] Connection header received
0810 18:48:48.816321 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:48.816326 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:48.816498 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:48.816640 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:48.816910 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.817693 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.818799 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.819607 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.819738 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:48.819855 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:48.819944 (+ 89us) server_negotiation.cc:299] Negotiation successful
0810 18:48:48.820036 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":118,"threads_started":1}
I20260810 18:48:48.821307 28248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:48.815071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51332 (local address 127.18.168.129:43315)
0810 18:48:48.815481 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:48.815487 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:48.815510 (+ 23us) server_negotiation.cc:407] Connection header received
0810 18:48:48.815599 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:48.815604 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:48.815748 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:48.815902 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:48.816182 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.817143 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.819307 (+ 2164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.820098 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.820235 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:48.820343 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:48.821062 (+ 719us) server_negotiation.cc:299] Negotiation successful
0810 18:48:48.821157 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":90,"threads_started":1}
I20260810 18:48:48.823426 27963 tablet_service.cc:1467] Processing CreateTablet for tablet e21d32c329ba4c429a0367c794cc7dc8 (DEFAULT_TABLE table=test-table [id=dc6e1c8b2e9649609d1367e66192f035]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:48.823757 27800 tablet_service.cc:1467] Processing CreateTablet for tablet e21d32c329ba4c429a0367c794cc7dc8 (DEFAULT_TABLE table=test-table [id=dc6e1c8b2e9649609d1367e66192f035]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:48.824522 27963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e21d32c329ba4c429a0367c794cc7dc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:48.824339 28132 tablet_service.cc:1467] Processing CreateTablet for tablet e21d32c329ba4c429a0367c794cc7dc8 (DEFAULT_TABLE table=test-table [id=dc6e1c8b2e9649609d1367e66192f035]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:48.824776 27800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e21d32c329ba4c429a0367c794cc7dc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:48.825301 28132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e21d32c329ba4c429a0367c794cc7dc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:48.831781 28249 tablet_bootstrap.cc:492] T e21d32c329ba4c429a0367c794cc7dc8 P 7895952636a34ed7b8d53efba58d8dbb: Bootstrap starting.
I20260810 18:48:48.832439 28250 tablet_bootstrap.cc:492] T e21d32c329ba4c429a0367c794cc7dc8 P 87492194659641e2a9141f7233bcc623: Bootstrap starting.
I20260810 18:48:48.833026 28253 tablet_bootstrap.cc:492] T e21d32c329ba4c429a0367c794cc7dc8 P a07e4e0445a140de92c89b90c930e418: Bootstrap starting.
I20260810 18:48:48.834342 28250 tablet_bootstrap.cc:654] T e21d32c329ba4c429a0367c794cc7dc8 P 87492194659641e2a9141f7233bcc623: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:48.834549 28249 tablet_bootstrap.cc:654] T e21d32c329ba4c429a0367c794cc7dc8 P 7895952636a34ed7b8d53efba58d8dbb: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:48.834977 28253 tablet_bootstrap.cc:654] T e21d32c329ba4c429a0367c794cc7dc8 P a07e4e0445a140de92c89b90c930e418: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:48.835346 28250 log.cc:824] T e21d32c329ba4c429a0367c794cc7dc8 P 87492194659641e2a9141f7233bcc623: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:48.835484 28249 log.cc:824] T e21d32c329ba4c429a0367c794cc7dc8 P 7895952636a34ed7b8d53efba58d8dbb: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:48.835737 28253 log.cc:824] T e21d32c329ba4c429a0367c794cc7dc8 P a07e4e0445a140de92c89b90c930e418: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:48.837455 28250 tablet_bootstrap.cc:492] T e21d32c329ba4c429a0367c794cc7dc8 P 87492194659641e2a9141f7233bcc623: No bootstrap required, opened a new log
I20260810 18:48:48.837455 28249 tablet_bootstrap.cc:492] T e21d32c329ba4c429a0367c794cc7dc8 P 7895952636a34ed7b8d53efba58d8dbb: No bootstrap required, opened a new log
I20260810 18:48:48.837579 28253 tablet_bootstrap.cc:492] T e21d32c329ba4c429a0367c794cc7dc8 P a07e4e0445a140de92c89b90c930e418: No bootstrap required, opened a new log
I20260810 18:48:48.837644 28249 ts_tablet_manager.cc:1397] T e21d32c329ba4c429a0367c794cc7dc8 P 7895952636a34ed7b8d53efba58d8dbb: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 18:48:48.837644 28250 ts_tablet_manager.cc:1397] T e21d32c329ba4c429a0367c794cc7dc8 P 87492194659641e2a9141f7233bcc623: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260810 18:48:48.837706 28253 ts_tablet_manager.cc:1397] T e21d32c329ba4c429a0367c794cc7dc8 P a07e4e0445a140de92c89b90c930e418: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 18:48:48.840449 28253 raft_consensus.cc:348] T e21d32c329ba4c429a0367c794cc7dc8 P a07e4e0445a140de92c89b90c930e418 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } }
I20260810 18:48:48.840636 28253 raft_consensus.cc:374] T e21d32c329ba4c429a0367c794cc7dc8 P a07e4e0445a140de92c89b90c930e418 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:48.840669 28253 raft_consensus.cc:729] T e21d32c329ba4c429a0367c794cc7dc8 P a07e4e0445a140de92c89b90c930e418 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a07e4e0445a140de92c89b90c930e418, State: Initialized, Role: FOLLOWER
I20260810 18:48:48.841158 28253 consensus_queue.cc:260] T e21d32c329ba4c429a0367c794cc7dc8 P a07e4e0445a140de92c89b90c930e418 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } }
I20260810 18:48:48.841593 28249 raft_consensus.cc:348] T e21d32c329ba4c429a0367c794cc7dc8 P 7895952636a34ed7b8d53efba58d8dbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } }
I20260810 18:48:48.841828 28217 heartbeater.cc:503] Master 127.18.168.190:43829 was elected leader, sending a full tablet report...
I20260810 18:48:48.841831 28249 raft_consensus.cc:374] T e21d32c329ba4c429a0367c794cc7dc8 P 7895952636a34ed7b8d53efba58d8dbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:48.841886 28249 raft_consensus.cc:729] T e21d32c329ba4c429a0367c794cc7dc8 P 7895952636a34ed7b8d53efba58d8dbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7895952636a34ed7b8d53efba58d8dbb, State: Initialized, Role: FOLLOWER
I20260810 18:48:48.842197 28253 ts_tablet_manager.cc:1428] T e21d32c329ba4c429a0367c794cc7dc8 P a07e4e0445a140de92c89b90c930e418: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260810 18:48:48.842332 28249 consensus_queue.cc:260] T e21d32c329ba4c429a0367c794cc7dc8 P 7895952636a34ed7b8d53efba58d8dbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } }
I20260810 18:48:48.841593 28250 raft_consensus.cc:348] T e21d32c329ba4c429a0367c794cc7dc8 P 87492194659641e2a9141f7233bcc623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } }
I20260810 18:48:48.842844 28250 raft_consensus.cc:374] T e21d32c329ba4c429a0367c794cc7dc8 P 87492194659641e2a9141f7233bcc623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:48.842904 28250 raft_consensus.cc:729] T e21d32c329ba4c429a0367c794cc7dc8 P 87492194659641e2a9141f7233bcc623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87492194659641e2a9141f7233bcc623, State: Initialized, Role: FOLLOWER
I20260810 18:48:48.843174 28044 heartbeater.cc:503] Master 127.18.168.190:43829 was elected leader, sending a full tablet report...
I20260810 18:48:48.843277 28250 consensus_queue.cc:260] T e21d32c329ba4c429a0367c794cc7dc8 P 87492194659641e2a9141f7233bcc623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } }
I20260810 18:48:48.843868 27876 heartbeater.cc:503] Master 127.18.168.190:43829 was elected leader, sending a full tablet report...
I20260810 18:48:48.844233 28250 ts_tablet_manager.cc:1428] T e21d32c329ba4c429a0367c794cc7dc8 P 87492194659641e2a9141f7233bcc623: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260810 18:48:48.845124 28249 ts_tablet_manager.cc:1428] T e21d32c329ba4c429a0367c794cc7dc8 P 7895952636a34ed7b8d53efba58d8dbb: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260810 18:48:48.861171 28258 raft_consensus.cc:482] T e21d32c329ba4c429a0367c794cc7dc8 P a07e4e0445a140de92c89b90c930e418 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:48:48.861378 28258 raft_consensus.cc:504] T e21d32c329ba4c429a0367c794cc7dc8 P a07e4e0445a140de92c89b90c930e418 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } }
I20260810 18:48:48.862484 28258 leader_election.cc:288] T e21d32c329ba4c429a0367c794cc7dc8 P a07e4e0445a140de92c89b90c930e418 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7895952636a34ed7b8d53efba58d8dbb (127.18.168.130:37203), 87492194659641e2a9141f7233bcc623 (127.18.168.129:43315)
I20260810 18:48:48.866309 28081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:48.862661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:37203 (local address 127.18.168.131:44277)
0810 18:48:48.862881 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:48.862897 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:48.862988 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:48.863282 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:48.863285 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:48.863305 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:48.863443 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:48.863449 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.864234 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.864237 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:48.865269 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:48.865281 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.865959 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.865962 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:48.866039 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:48.866059 (+ 20us) client_negotiation.cc:769] Sending connection context
0810 18:48:48.866135 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 18:48:48.866199 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":18}
I20260810 18:48:48.866376 28244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:48.862796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:44277 (local address 127.18.168.130:37203)
0810 18:48:48.862926 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:48.862932 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:48.862948 (+ 16us) server_negotiation.cc:407] Connection header received
0810 18:48:48.863104 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:48.863107 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:48.863147 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:48.863248 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:48.863569 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.864114 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.865393 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.865794 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.865925 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:48.865976 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:48.866179 (+ 203us) server_negotiation.cc:299] Negotiation successful
0810 18:48:48.866262 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 18:48:48.867029 28248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:48.862840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:36293 (local address 127.18.168.129:43315)
0810 18:48:48.862964 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:48.862968 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:48.863202 (+ 234us) server_negotiation.cc:407] Connection header received
0810 18:48:48.863295 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:48.863298 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:48.863347 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:48.863417 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:48.864136 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.864672 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.865912 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.866491 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.866606 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:48.866640 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:48.866858 (+ 218us) server_negotiation.cc:299] Negotiation successful
0810 18:48:48.866926 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 18:48:48.867038 28261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:48.862748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:43315 (local address 127.18.168.131:36293)
0810 18:48:48.863101 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:48.863113 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:48.863199 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:48.863449 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:48.863453 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:48.863468 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:48.863992 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:48.864000 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.864841 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.864845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:48.865793 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:48.865802 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.866601 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.866605 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:48.866714 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:48.866737 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 18:48:48.866815 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 18:48:48.866886 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"mutex_wait_us":10,"thread_start_us":134,"threads_started":1}
I20260810 18:48:48.867115 27987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e21d32c329ba4c429a0367c794cc7dc8" candidate_uuid: "a07e4e0445a140de92c89b90c930e418" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7895952636a34ed7b8d53efba58d8dbb" is_pre_election: true
I20260810 18:48:48.867439 27987 raft_consensus.cc:2454] T e21d32c329ba4c429a0367c794cc7dc8 P 7895952636a34ed7b8d53efba58d8dbb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a07e4e0445a140de92c89b90c930e418 in term 0.
I20260810 18:48:48.867763 27823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e21d32c329ba4c429a0367c794cc7dc8" candidate_uuid: "a07e4e0445a140de92c89b90c930e418" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87492194659641e2a9141f7233bcc623" is_pre_election: true
I20260810 18:48:48.867964 28074 leader_election.cc:302] T e21d32c329ba4c429a0367c794cc7dc8 P a07e4e0445a140de92c89b90c930e418 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7895952636a34ed7b8d53efba58d8dbb, a07e4e0445a140de92c89b90c930e418; no voters:
I20260810 18:48:48.868019 27823 raft_consensus.cc:2454] T e21d32c329ba4c429a0367c794cc7dc8 P 87492194659641e2a9141f7233bcc623 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a07e4e0445a140de92c89b90c930e418 in term 0.
I20260810 18:48:48.868227 28258 raft_consensus.cc:2781] T e21d32c329ba4c429a0367c794cc7dc8 P a07e4e0445a140de92c89b90c930e418 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:48:48.868314 28258 raft_consensus.cc:482] T e21d32c329ba4c429a0367c794cc7dc8 P a07e4e0445a140de92c89b90c930e418 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:48:48.868386 28258 raft_consensus.cc:3037] T e21d32c329ba4c429a0367c794cc7dc8 P a07e4e0445a140de92c89b90c930e418 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:48.869448 28258 raft_consensus.cc:504] T e21d32c329ba4c429a0367c794cc7dc8 P a07e4e0445a140de92c89b90c930e418 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } }
I20260810 18:48:48.869850 28258 leader_election.cc:288] T e21d32c329ba4c429a0367c794cc7dc8 P a07e4e0445a140de92c89b90c930e418 [CANDIDATE]: Term 1 election: Requested vote from peers 7895952636a34ed7b8d53efba58d8dbb (127.18.168.130:37203), 87492194659641e2a9141f7233bcc623 (127.18.168.129:43315)
I20260810 18:48:48.870193 27987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e21d32c329ba4c429a0367c794cc7dc8" candidate_uuid: "a07e4e0445a140de92c89b90c930e418" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7895952636a34ed7b8d53efba58d8dbb"
I20260810 18:48:48.870301 27823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e21d32c329ba4c429a0367c794cc7dc8" candidate_uuid: "a07e4e0445a140de92c89b90c930e418" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87492194659641e2a9141f7233bcc623"
I20260810 18:48:48.870313 27987 raft_consensus.cc:3037] T e21d32c329ba4c429a0367c794cc7dc8 P 7895952636a34ed7b8d53efba58d8dbb [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:48.870414 27823 raft_consensus.cc:3037] T e21d32c329ba4c429a0367c794cc7dc8 P 87492194659641e2a9141f7233bcc623 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:48.871438 27823 raft_consensus.cc:2454] T e21d32c329ba4c429a0367c794cc7dc8 P 87492194659641e2a9141f7233bcc623 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a07e4e0445a140de92c89b90c930e418 in term 1.
I20260810 18:48:48.871767 27987 raft_consensus.cc:2454] T e21d32c329ba4c429a0367c794cc7dc8 P 7895952636a34ed7b8d53efba58d8dbb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a07e4e0445a140de92c89b90c930e418 in term 1.
I20260810 18:48:48.871790 28071 leader_election.cc:302] T e21d32c329ba4c429a0367c794cc7dc8 P a07e4e0445a140de92c89b90c930e418 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87492194659641e2a9141f7233bcc623, a07e4e0445a140de92c89b90c930e418; no voters:
I20260810 18:48:48.871953 28258 raft_consensus.cc:2781] T e21d32c329ba4c429a0367c794cc7dc8 P a07e4e0445a140de92c89b90c930e418 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:48.872189 28258 raft_consensus.cc:686] T e21d32c329ba4c429a0367c794cc7dc8 P a07e4e0445a140de92c89b90c930e418 [term 1 LEADER]: Becoming Leader. State: Replica: a07e4e0445a140de92c89b90c930e418, State: Running, Role: LEADER
I20260810 18:48:48.872522 28258 consensus_queue.cc:237] T e21d32c329ba4c429a0367c794cc7dc8 P a07e4e0445a140de92c89b90c930e418 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } }
I20260810 18:48:48.875435 27643 catalog_manager.cc:5495] T e21d32c329ba4c429a0367c794cc7dc8 P a07e4e0445a140de92c89b90c930e418 reported cstate change: term changed from 0 to 1, leader changed from <none> to a07e4e0445a140de92c89b90c930e418 (127.18.168.131). New cstate: current_term: 1 leader_uuid: "a07e4e0445a140de92c89b90c930e418" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } health_report { overall_health: HEALTHY } } }
I20260810 18:48:48.897289 28246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:48.893773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39234 (local address 127.18.168.131:43327)
0810 18:48:48.893893 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:48.893897 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:48.893920 (+ 23us) server_negotiation.cc:407] Connection header received
0810 18:48:48.894040 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:48.894043 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:48.894113 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:48.894198 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:48:48.894455 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.895022 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.895923 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.896134 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.896211 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:48.896437 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:48:48.896925 (+ 488us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:48:48.897010 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:48.897076 (+ 66us) server_negotiation.cc:299] Negotiation successful
0810 18:48:48.897124 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 18:48:48.903167 27823 raft_consensus.cc:1261] T e21d32c329ba4c429a0367c794cc7dc8 P 87492194659641e2a9141f7233bcc623 [term 1 FOLLOWER]: Refusing update from remote peer a07e4e0445a140de92c89b90c930e418: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:48.903167 27987 raft_consensus.cc:1261] T e21d32c329ba4c429a0367c794cc7dc8 P 7895952636a34ed7b8d53efba58d8dbb [term 1 FOLLOWER]: Refusing update from remote peer a07e4e0445a140de92c89b90c930e418: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:48.903848 28263 consensus_queue.cc:1035] T e21d32c329ba4c429a0367c794cc7dc8 P a07e4e0445a140de92c89b90c930e418 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:48.904064 28258 consensus_queue.cc:1035] T e21d32c329ba4c429a0367c794cc7dc8 P a07e4e0445a140de92c89b90c930e418 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:48.920969 27642 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:48.921118 27642 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:48.923385 27642 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dfbc86f800924b13a83b5a0e342be613: Sending DeleteTablet for 3 replicas of tablet e21d32c329ba4c429a0367c794cc7dc8
I20260810 18:48:48.924296 28132 tablet_service.cc:1514] Processing DeleteTablet for tablet e21d32c329ba4c429a0367c794cc7dc8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:48 UTC) from {username='oryx', principal='oryx/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:39218
I20260810 18:48:48.924296 27800 tablet_service.cc:1514] Processing DeleteTablet for tablet e21d32c329ba4c429a0367c794cc7dc8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:48 UTC) from {username='oryx', principal='oryx/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:51332
I20260810 18:48:48.924301 27963 tablet_service.cc:1514] Processing DeleteTablet for tablet e21d32c329ba4c429a0367c794cc7dc8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:48 UTC) from {username='oryx', principal='oryx/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:39090
I20260810 18:48:48.924912 28279 tablet_replica.cc:330] T e21d32c329ba4c429a0367c794cc7dc8 P 87492194659641e2a9141f7233bcc623: stopping tablet replica
I20260810 18:48:48.924975 27642 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52562:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 18:48:48.925244 28279 raft_consensus.cc:2229] T e21d32c329ba4c429a0367c794cc7dc8 P 87492194659641e2a9141f7233bcc623 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260810 18:48:48.925429 27642 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:48:48.925565 28279 raft_consensus.cc:2258] T e21d32c329ba4c429a0367c794cc7dc8 P 87492194659641e2a9141f7233bcc623 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:48.925920 28280 tablet_replica.cc:330] T e21d32c329ba4c429a0367c794cc7dc8 P a07e4e0445a140de92c89b90c930e418: stopping tablet replica
I20260810 18:48:48.926206 28280 raft_consensus.cc:2229] T e21d32c329ba4c429a0367c794cc7dc8 P a07e4e0445a140de92c89b90c930e418 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:48:48.926579 28280 raft_consensus.cc:2258] T e21d32c329ba4c429a0367c794cc7dc8 P a07e4e0445a140de92c89b90c930e418 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:48.928156 28280 ts_tablet_manager.cc:1905] T e21d32c329ba4c429a0367c794cc7dc8 P a07e4e0445a140de92c89b90c930e418: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:48.930480 27800 tablet_service.cc:1467] Processing CreateTablet for tablet 7fd2b1de41094b10bff247a31a19f6c6 (DEFAULT_TABLE table=test-table [id=2670a64ab2ed4c0ab8e9494daa4a5344]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:48.930645 27963 tablet_service.cc:1467] Processing CreateTablet for tablet 7fd2b1de41094b10bff247a31a19f6c6 (DEFAULT_TABLE table=test-table [id=2670a64ab2ed4c0ab8e9494daa4a5344]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:48.930680 28132 tablet_service.cc:1467] Processing CreateTablet for tablet 7fd2b1de41094b10bff247a31a19f6c6 (DEFAULT_TABLE table=test-table [id=2670a64ab2ed4c0ab8e9494daa4a5344]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:48.930748 27800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fd2b1de41094b10bff247a31a19f6c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:48.930984 27963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fd2b1de41094b10bff247a31a19f6c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:48.933185 28250 tablet_bootstrap.cc:492] T 7fd2b1de41094b10bff247a31a19f6c6 P 87492194659641e2a9141f7233bcc623: Bootstrap starting.
I20260810 18:48:48.933558 28249 tablet_bootstrap.cc:492] T 7fd2b1de41094b10bff247a31a19f6c6 P 7895952636a34ed7b8d53efba58d8dbb: Bootstrap starting.
I20260810 18:48:48.934172 28250 tablet_bootstrap.cc:654] T 7fd2b1de41094b10bff247a31a19f6c6 P 87492194659641e2a9141f7233bcc623: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:48.934233 28279 ts_tablet_manager.cc:1905] T e21d32c329ba4c429a0367c794cc7dc8 P 87492194659641e2a9141f7233bcc623: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:48.934574 28249 tablet_bootstrap.cc:654] T 7fd2b1de41094b10bff247a31a19f6c6 P 7895952636a34ed7b8d53efba58d8dbb: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:48.936565 28249 tablet_bootstrap.cc:492] T 7fd2b1de41094b10bff247a31a19f6c6 P 7895952636a34ed7b8d53efba58d8dbb: No bootstrap required, opened a new log
I20260810 18:48:48.936560 28279 ts_tablet_manager.cc:1918] T e21d32c329ba4c429a0367c794cc7dc8 P 87492194659641e2a9141f7233bcc623: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:48:48.936645 28279 log.cc:1196] T e21d32c329ba4c429a0367c794cc7dc8 P 87492194659641e2a9141f7233bcc623: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-0/wal/wals/e21d32c329ba4c429a0367c794cc7dc8
I20260810 18:48:48.936666 28249 ts_tablet_manager.cc:1397] T 7fd2b1de41094b10bff247a31a19f6c6 P 7895952636a34ed7b8d53efba58d8dbb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 18:48:48.936827 28250 tablet_bootstrap.cc:492] T 7fd2b1de41094b10bff247a31a19f6c6 P 87492194659641e2a9141f7233bcc623: No bootstrap required, opened a new log
I20260810 18:48:48.936896 28250 ts_tablet_manager.cc:1397] T 7fd2b1de41094b10bff247a31a19f6c6 P 87492194659641e2a9141f7233bcc623: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260810 18:48:48.937235 28249 raft_consensus.cc:348] T 7fd2b1de41094b10bff247a31a19f6c6 P 7895952636a34ed7b8d53efba58d8dbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } }
I20260810 18:48:48.937340 28249 raft_consensus.cc:374] T 7fd2b1de41094b10bff247a31a19f6c6 P 7895952636a34ed7b8d53efba58d8dbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:48.937379 28249 raft_consensus.cc:729] T 7fd2b1de41094b10bff247a31a19f6c6 P 7895952636a34ed7b8d53efba58d8dbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7895952636a34ed7b8d53efba58d8dbb, State: Initialized, Role: FOLLOWER
I20260810 18:48:48.937361 28250 raft_consensus.cc:348] T 7fd2b1de41094b10bff247a31a19f6c6 P 87492194659641e2a9141f7233bcc623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } }
I20260810 18:48:48.937455 28250 raft_consensus.cc:374] T 7fd2b1de41094b10bff247a31a19f6c6 P 87492194659641e2a9141f7233bcc623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:48.937486 28250 raft_consensus.cc:729] T 7fd2b1de41094b10bff247a31a19f6c6 P 87492194659641e2a9141f7233bcc623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87492194659641e2a9141f7233bcc623, State: Initialized, Role: FOLLOWER
I20260810 18:48:48.937507 28249 consensus_queue.cc:260] T 7fd2b1de41094b10bff247a31a19f6c6 P 7895952636a34ed7b8d53efba58d8dbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } }
I20260810 18:48:48.937592 28250 consensus_queue.cc:260] T 7fd2b1de41094b10bff247a31a19f6c6 P 87492194659641e2a9141f7233bcc623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } }
I20260810 18:48:48.937801 28249 ts_tablet_manager.cc:1428] T 7fd2b1de41094b10bff247a31a19f6c6 P 7895952636a34ed7b8d53efba58d8dbb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 18:48:48.937831 28250 ts_tablet_manager.cc:1428] T 7fd2b1de41094b10bff247a31a19f6c6 P 87492194659641e2a9141f7233bcc623: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 18:48:48.938421 28132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fd2b1de41094b10bff247a31a19f6c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:48.939090 28279 ts_tablet_manager.cc:1939] T e21d32c329ba4c429a0367c794cc7dc8 P 87492194659641e2a9141f7233bcc623: Deleting consensus metadata
I20260810 18:48:48.940224 27625 catalog_manager.cc:4841] TS 87492194659641e2a9141f7233bcc623 (127.18.168.129:43315): tablet e21d32c329ba4c429a0367c794cc7dc8 (table test-table [id=dc6e1c8b2e9649609d1367e66192f035]) successfully deleted
I20260810 18:48:48.940652 28280 ts_tablet_manager.cc:1918] T e21d32c329ba4c429a0367c794cc7dc8 P a07e4e0445a140de92c89b90c930e418: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:48:48.940757 28280 log.cc:1196] T e21d32c329ba4c429a0367c794cc7dc8 P a07e4e0445a140de92c89b90c930e418: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-2/wal/wals/e21d32c329ba4c429a0367c794cc7dc8
I20260810 18:48:48.940853 28253 tablet_bootstrap.cc:492] T 7fd2b1de41094b10bff247a31a19f6c6 P a07e4e0445a140de92c89b90c930e418: Bootstrap starting.
I20260810 18:48:48.941203 28280 ts_tablet_manager.cc:1939] T e21d32c329ba4c429a0367c794cc7dc8 P a07e4e0445a140de92c89b90c930e418: Deleting consensus metadata
I20260810 18:48:48.941713 28253 tablet_bootstrap.cc:654] T 7fd2b1de41094b10bff247a31a19f6c6 P a07e4e0445a140de92c89b90c930e418: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:48.942234 27625 catalog_manager.cc:4841] TS a07e4e0445a140de92c89b90c930e418 (127.18.168.131:43327): tablet e21d32c329ba4c429a0367c794cc7dc8 (table test-table [id=dc6e1c8b2e9649609d1367e66192f035]) successfully deleted
I20260810 18:48:48.942283 28281 tablet_replica.cc:330] T e21d32c329ba4c429a0367c794cc7dc8 P 7895952636a34ed7b8d53efba58d8dbb: stopping tablet replica
I20260810 18:48:48.942677 28281 raft_consensus.cc:2229] T e21d32c329ba4c429a0367c794cc7dc8 P 7895952636a34ed7b8d53efba58d8dbb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:48.942965 28281 raft_consensus.cc:2258] T e21d32c329ba4c429a0367c794cc7dc8 P 7895952636a34ed7b8d53efba58d8dbb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:48.944386 28281 ts_tablet_manager.cc:1905] T e21d32c329ba4c429a0367c794cc7dc8 P 7895952636a34ed7b8d53efba58d8dbb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:48.945500 28253 tablet_bootstrap.cc:492] T 7fd2b1de41094b10bff247a31a19f6c6 P a07e4e0445a140de92c89b90c930e418: No bootstrap required, opened a new log
I20260810 18:48:48.945602 28253 ts_tablet_manager.cc:1397] T 7fd2b1de41094b10bff247a31a19f6c6 P a07e4e0445a140de92c89b90c930e418: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260810 18:48:48.945990 28253 raft_consensus.cc:348] T 7fd2b1de41094b10bff247a31a19f6c6 P a07e4e0445a140de92c89b90c930e418 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } }
I20260810 18:48:48.946103 28253 raft_consensus.cc:374] T 7fd2b1de41094b10bff247a31a19f6c6 P a07e4e0445a140de92c89b90c930e418 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:48.946142 28253 raft_consensus.cc:729] T 7fd2b1de41094b10bff247a31a19f6c6 P a07e4e0445a140de92c89b90c930e418 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a07e4e0445a140de92c89b90c930e418, State: Initialized, Role: FOLLOWER
I20260810 18:48:48.946305 28253 consensus_queue.cc:260] T 7fd2b1de41094b10bff247a31a19f6c6 P a07e4e0445a140de92c89b90c930e418 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } }
I20260810 18:48:48.946867 28281 ts_tablet_manager.cc:1918] T e21d32c329ba4c429a0367c794cc7dc8 P 7895952636a34ed7b8d53efba58d8dbb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:48:48.946956 28281 log.cc:1196] T e21d32c329ba4c429a0367c794cc7dc8 P 7895952636a34ed7b8d53efba58d8dbb: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-1/wal/wals/e21d32c329ba4c429a0367c794cc7dc8
I20260810 18:48:48.947252 28281 ts_tablet_manager.cc:1939] T e21d32c329ba4c429a0367c794cc7dc8 P 7895952636a34ed7b8d53efba58d8dbb: Deleting consensus metadata
I20260810 18:48:48.947723 28253 ts_tablet_manager.cc:1428] T 7fd2b1de41094b10bff247a31a19f6c6 P a07e4e0445a140de92c89b90c930e418: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260810 18:48:48.948268 27627 catalog_manager.cc:4841] TS 7895952636a34ed7b8d53efba58d8dbb (127.18.168.130:37203): tablet e21d32c329ba4c429a0367c794cc7dc8 (table test-table [id=dc6e1c8b2e9649609d1367e66192f035]) successfully deleted
W20260810 18:48:49.006608 28218 tablet.cc:2369] T 7fd2b1de41094b10bff247a31a19f6c6 P a07e4e0445a140de92c89b90c930e418: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:48:49.042198 28045 tablet.cc:2369] T 7fd2b1de41094b10bff247a31a19f6c6 P 7895952636a34ed7b8d53efba58d8dbb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:48:49.063858 27877 tablet.cc:2369] T 7fd2b1de41094b10bff247a31a19f6c6 P 87492194659641e2a9141f7233bcc623: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:48:49.103727 28260 raft_consensus.cc:482] T 7fd2b1de41094b10bff247a31a19f6c6 P 87492194659641e2a9141f7233bcc623 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:48:49.103868 28260 raft_consensus.cc:504] T 7fd2b1de41094b10bff247a31a19f6c6 P 87492194659641e2a9141f7233bcc623 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } }
I20260810 18:48:49.105083 28260 leader_election.cc:288] T 7fd2b1de41094b10bff247a31a19f6c6 P 87492194659641e2a9141f7233bcc623 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7895952636a34ed7b8d53efba58d8dbb (127.18.168.130:37203), a07e4e0445a140de92c89b90c930e418 (127.18.168.131:43327)
I20260810 18:48:49.110399 28246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:49.105428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:54481 (local address 127.18.168.131:43327)
0810 18:48:49.105559 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:49.105563 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:49.105778 (+ 215us) server_negotiation.cc:407] Connection header received
0810 18:48:49.105890 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:49.105892 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:49.105940 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:49.106011 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:49.106473 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.107305 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.109022 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.109604 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.109749 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:49.109804 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:49.110198 (+ 394us) server_negotiation.cc:299] Negotiation successful
0810 18:48:49.110268 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 18:48:49.110400 28244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:49.105374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:40925 (local address 127.18.168.130:37203)
0810 18:48:49.105512 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:49.105517 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:49.105699 (+ 182us) server_negotiation.cc:407] Connection header received
0810 18:48:49.105860 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:49.105865 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:49.105919 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:49.106004 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:49.106473 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.107282 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.109022 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.109604 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.109749 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:49.109804 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:49.110198 (+ 394us) server_negotiation.cc:299] Negotiation successful
0810 18:48:49.110268 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 18:48:49.110479 28287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:49.105379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.131:43327 (local address 127.18.168.129:54481)
0810 18:48:49.105694 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:49.105706 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:49.105792 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:49.106080 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:49.106083 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:49.106131 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:49.106348 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:49.106355 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.107423 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.107427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:49.108884 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:49.108895 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.109777 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.109781 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:49.110004 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:49.110091 (+ 87us) client_negotiation.cc:769] Sending connection context
0810 18:48:49.110162 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 18:48:49.110283 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":216,"spinlock_wait_cycles":11392,"thread_start_us":112,"threads_started":1}
I20260810 18:48:49.110479 28286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:49.105248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:37203 (local address 127.18.168.129:40925)
0810 18:48:49.105578 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:49.105595 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:49.105734 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:49.106080 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:49.106083 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:49.106129 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:49.106347 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:49.106355 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.107423 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.107427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:49.108887 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:49.108895 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.109777 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.109781 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:49.109965 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:49.110079 (+ 114us) client_negotiation.cc:769] Sending connection context
0810 18:48:49.110162 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 18:48:49.110281 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":267,"spinlock_wait_cycles":31744,"thread_start_us":75,"threads_started":1}
I20260810 18:48:49.111056 27987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fd2b1de41094b10bff247a31a19f6c6" candidate_uuid: "87492194659641e2a9141f7233bcc623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7895952636a34ed7b8d53efba58d8dbb" is_pre_election: true
I20260810 18:48:49.111243 27987 raft_consensus.cc:2454] T 7fd2b1de41094b10bff247a31a19f6c6 P 7895952636a34ed7b8d53efba58d8dbb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87492194659641e2a9141f7233bcc623 in term 0.
I20260810 18:48:49.111548 28153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fd2b1de41094b10bff247a31a19f6c6" candidate_uuid: "87492194659641e2a9141f7233bcc623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a07e4e0445a140de92c89b90c930e418" is_pre_election: true
I20260810 18:48:49.111651 27744 leader_election.cc:302] T 7fd2b1de41094b10bff247a31a19f6c6 P 87492194659641e2a9141f7233bcc623 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7895952636a34ed7b8d53efba58d8dbb, 87492194659641e2a9141f7233bcc623; no voters:
I20260810 18:48:49.111837 28153 raft_consensus.cc:2454] T 7fd2b1de41094b10bff247a31a19f6c6 P a07e4e0445a140de92c89b90c930e418 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87492194659641e2a9141f7233bcc623 in term 0.
I20260810 18:48:49.111975 28260 raft_consensus.cc:2781] T 7fd2b1de41094b10bff247a31a19f6c6 P 87492194659641e2a9141f7233bcc623 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:48:49.112066 28260 raft_consensus.cc:482] T 7fd2b1de41094b10bff247a31a19f6c6 P 87492194659641e2a9141f7233bcc623 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:48:49.112130 28260 raft_consensus.cc:3037] T 7fd2b1de41094b10bff247a31a19f6c6 P 87492194659641e2a9141f7233bcc623 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:49.113240 28260 raft_consensus.cc:504] T 7fd2b1de41094b10bff247a31a19f6c6 P 87492194659641e2a9141f7233bcc623 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } }
I20260810 18:48:49.113633 28260 leader_election.cc:288] T 7fd2b1de41094b10bff247a31a19f6c6 P 87492194659641e2a9141f7233bcc623 [CANDIDATE]: Term 1 election: Requested vote from peers 7895952636a34ed7b8d53efba58d8dbb (127.18.168.130:37203), a07e4e0445a140de92c89b90c930e418 (127.18.168.131:43327)
I20260810 18:48:49.113960 27987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fd2b1de41094b10bff247a31a19f6c6" candidate_uuid: "87492194659641e2a9141f7233bcc623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7895952636a34ed7b8d53efba58d8dbb"
I20260810 18:48:49.113960 28153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fd2b1de41094b10bff247a31a19f6c6" candidate_uuid: "87492194659641e2a9141f7233bcc623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a07e4e0445a140de92c89b90c930e418"
I20260810 18:48:49.114085 27987 raft_consensus.cc:3037] T 7fd2b1de41094b10bff247a31a19f6c6 P 7895952636a34ed7b8d53efba58d8dbb [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:49.114085 28153 raft_consensus.cc:3037] T 7fd2b1de41094b10bff247a31a19f6c6 P a07e4e0445a140de92c89b90c930e418 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:49.115293 27987 raft_consensus.cc:2454] T 7fd2b1de41094b10bff247a31a19f6c6 P 7895952636a34ed7b8d53efba58d8dbb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87492194659641e2a9141f7233bcc623 in term 1.
I20260810 18:48:49.115293 28153 raft_consensus.cc:2454] T 7fd2b1de41094b10bff247a31a19f6c6 P a07e4e0445a140de92c89b90c930e418 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87492194659641e2a9141f7233bcc623 in term 1.
I20260810 18:48:49.115578 27744 leader_election.cc:302] T 7fd2b1de41094b10bff247a31a19f6c6 P 87492194659641e2a9141f7233bcc623 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7895952636a34ed7b8d53efba58d8dbb, 87492194659641e2a9141f7233bcc623; no voters:
I20260810 18:48:49.115796 28260 raft_consensus.cc:2781] T 7fd2b1de41094b10bff247a31a19f6c6 P 87492194659641e2a9141f7233bcc623 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:49.116026 28260 raft_consensus.cc:686] T 7fd2b1de41094b10bff247a31a19f6c6 P 87492194659641e2a9141f7233bcc623 [term 1 LEADER]: Becoming Leader. State: Replica: 87492194659641e2a9141f7233bcc623, State: Running, Role: LEADER
I20260810 18:48:49.116276 28260 consensus_queue.cc:237] T 7fd2b1de41094b10bff247a31a19f6c6 P 87492194659641e2a9141f7233bcc623 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } }
I20260810 18:48:49.118683 27642 catalog_manager.cc:5495] T 7fd2b1de41094b10bff247a31a19f6c6 P 87492194659641e2a9141f7233bcc623 reported cstate change: term changed from 0 to 1, leader changed from <none> to 87492194659641e2a9141f7233bcc623 (127.18.168.129). New cstate: current_term: 1 leader_uuid: "87492194659641e2a9141f7233bcc623" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } health_report { overall_health: UNKNOWN } } }
Aug 10 18:48:49 dist-test-slave-cbsf krb5kdc[27593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387727, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.168.190@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM
I20260810 18:48:49.163512 28295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:49.154407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43829 (local address 127.0.0.1:52578)
0810 18:48:49.154722 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:49.154738 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:49.154856 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:49.155154 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:49.155158 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:49.155977 (+ 819us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:49.156112 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:49.156122 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.157187 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.157190 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:49.157948 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:49.157958 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.158380 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.158382 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:49.158447 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:49.158951 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:49.158962 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:49.160492 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:49.162080 (+ 1588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:49.162088 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:49.162093 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:49.162322 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:49.162593 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:49.162595 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:49.162597 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:49.162682 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:49.163019 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:49.163024 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:49.163126 (+ 102us) client_negotiation.cc:769] Sending connection context
0810 18:48:49.163219 (+ 93us) client_negotiation.cc:240] Negotiation successful
0810 18:48:49.163339 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":129,"threads_started":1}
I20260810 18:48:49.163517 27700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:49.154502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52578 (local address 127.18.168.190:43829)
0810 18:48:49.154702 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:49.154707 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:49.154798 (+ 91us) server_negotiation.cc:407] Connection header received
0810 18:48:49.154978 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:49.154981 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:49.155053 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:49.155137 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:49.156261 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.157030 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.158063 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.158277 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.158361 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:49.160636 (+ 2275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:49.160639 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:49.160641 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:49.160662 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:49.161982 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:49.162415 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:49.162417 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:49.162419 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:49.162497 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:49.162760 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:49.162763 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:49.162764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:49.162928 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:49.163004 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:49.163276 (+ 272us) server_negotiation.cc:299] Negotiation successful
0810 18:48:49.163396 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":84}
I20260810 18:48:49.167310 27642 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:52578:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 18:48:49.168042 27642 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:48:49.172986 27800 tablet_service.cc:1467] Processing CreateTablet for tablet 2c26911d42324f4abe7b8efefd35489b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=35f2078bd7514509bcf0692285d9a349]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 18:48:49.173074 27963 tablet_service.cc:1467] Processing CreateTablet for tablet 2c26911d42324f4abe7b8efefd35489b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=35f2078bd7514509bcf0692285d9a349]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 18:48:49.173101 28132 tablet_service.cc:1467] Processing CreateTablet for tablet 2c26911d42324f4abe7b8efefd35489b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=35f2078bd7514509bcf0692285d9a349]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 18:48:49.173404 27800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c26911d42324f4abe7b8efefd35489b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:49.173404 27963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c26911d42324f4abe7b8efefd35489b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:49.173516 28132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c26911d42324f4abe7b8efefd35489b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:49.175951 28253 tablet_bootstrap.cc:492] T 2c26911d42324f4abe7b8efefd35489b P a07e4e0445a140de92c89b90c930e418: Bootstrap starting.
I20260810 18:48:49.176286 28250 tablet_bootstrap.cc:492] T 2c26911d42324f4abe7b8efefd35489b P 87492194659641e2a9141f7233bcc623: Bootstrap starting.
I20260810 18:48:49.176882 28253 tablet_bootstrap.cc:654] T 2c26911d42324f4abe7b8efefd35489b P a07e4e0445a140de92c89b90c930e418: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:49.177116 28250 tablet_bootstrap.cc:654] T 2c26911d42324f4abe7b8efefd35489b P 87492194659641e2a9141f7233bcc623: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:49.178140 28253 tablet_bootstrap.cc:492] T 2c26911d42324f4abe7b8efefd35489b P a07e4e0445a140de92c89b90c930e418: No bootstrap required, opened a new log
I20260810 18:48:49.178218 28253 ts_tablet_manager.cc:1397] T 2c26911d42324f4abe7b8efefd35489b P a07e4e0445a140de92c89b90c930e418: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 18:48:49.178421 28250 tablet_bootstrap.cc:492] T 2c26911d42324f4abe7b8efefd35489b P 87492194659641e2a9141f7233bcc623: No bootstrap required, opened a new log
I20260810 18:48:49.178519 28250 ts_tablet_manager.cc:1397] T 2c26911d42324f4abe7b8efefd35489b P 87492194659641e2a9141f7233bcc623: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260810 18:48:49.178663 28253 raft_consensus.cc:348] T 2c26911d42324f4abe7b8efefd35489b P a07e4e0445a140de92c89b90c930e418 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } } peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } }
I20260810 18:48:49.178766 28253 raft_consensus.cc:374] T 2c26911d42324f4abe7b8efefd35489b P a07e4e0445a140de92c89b90c930e418 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:49.178817 28253 raft_consensus.cc:729] T 2c26911d42324f4abe7b8efefd35489b P a07e4e0445a140de92c89b90c930e418 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a07e4e0445a140de92c89b90c930e418, State: Initialized, Role: FOLLOWER
I20260810 18:48:49.178946 28253 consensus_queue.cc:260] T 2c26911d42324f4abe7b8efefd35489b P a07e4e0445a140de92c89b90c930e418 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } } peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } }
I20260810 18:48:49.178998 28250 raft_consensus.cc:348] T 2c26911d42324f4abe7b8efefd35489b P 87492194659641e2a9141f7233bcc623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } } peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } }
I20260810 18:48:49.179144 28250 raft_consensus.cc:374] T 2c26911d42324f4abe7b8efefd35489b P 87492194659641e2a9141f7233bcc623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:49.179191 28250 raft_consensus.cc:729] T 2c26911d42324f4abe7b8efefd35489b P 87492194659641e2a9141f7233bcc623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87492194659641e2a9141f7233bcc623, State: Initialized, Role: FOLLOWER
I20260810 18:48:49.179234 28253 ts_tablet_manager.cc:1428] T 2c26911d42324f4abe7b8efefd35489b P a07e4e0445a140de92c89b90c930e418: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 18:48:49.179168 28258 tablet_replica.cc:439] T 2c26911d42324f4abe7b8efefd35489b P a07e4e0445a140de92c89b90c930e418: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } } peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } }
I20260810 18:48:49.179325 28250 consensus_queue.cc:260] T 2c26911d42324f4abe7b8efefd35489b P 87492194659641e2a9141f7233bcc623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } } peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } }
I20260810 18:48:49.179436 28258 tablet_replica.cc:442] T 2c26911d42324f4abe7b8efefd35489b P a07e4e0445a140de92c89b90c930e418: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:48:49.179567 28260 tablet_replica.cc:439] T 2c26911d42324f4abe7b8efefd35489b P 87492194659641e2a9141f7233bcc623: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } } peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } }
I20260810 18:48:49.179695 28260 tablet_replica.cc:442] T 2c26911d42324f4abe7b8efefd35489b P 87492194659641e2a9141f7233bcc623: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:48:49.179979 28249 tablet_bootstrap.cc:492] T 2c26911d42324f4abe7b8efefd35489b P 7895952636a34ed7b8d53efba58d8dbb: Bootstrap starting.
I20260810 18:48:49.180299 28250 ts_tablet_manager.cc:1428] T 2c26911d42324f4abe7b8efefd35489b P 87492194659641e2a9141f7233bcc623: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260810 18:48:49.180939 28249 tablet_bootstrap.cc:654] T 2c26911d42324f4abe7b8efefd35489b P 7895952636a34ed7b8d53efba58d8dbb: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:49.182245 28249 tablet_bootstrap.cc:492] T 2c26911d42324f4abe7b8efefd35489b P 7895952636a34ed7b8d53efba58d8dbb: No bootstrap required, opened a new log
I20260810 18:48:49.182335 28249 ts_tablet_manager.cc:1397] T 2c26911d42324f4abe7b8efefd35489b P 7895952636a34ed7b8d53efba58d8dbb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 18:48:49.182806 28249 raft_consensus.cc:348] T 2c26911d42324f4abe7b8efefd35489b P 7895952636a34ed7b8d53efba58d8dbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } } peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } }
I20260810 18:48:49.182925 28249 raft_consensus.cc:374] T 2c26911d42324f4abe7b8efefd35489b P 7895952636a34ed7b8d53efba58d8dbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:49.182961 28249 raft_consensus.cc:729] T 2c26911d42324f4abe7b8efefd35489b P 7895952636a34ed7b8d53efba58d8dbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7895952636a34ed7b8d53efba58d8dbb, State: Initialized, Role: FOLLOWER
I20260810 18:48:49.183077 28249 consensus_queue.cc:260] T 2c26911d42324f4abe7b8efefd35489b P 7895952636a34ed7b8d53efba58d8dbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } } peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } }
I20260810 18:48:49.183333 28249 ts_tablet_manager.cc:1428] T 2c26911d42324f4abe7b8efefd35489b P 7895952636a34ed7b8d53efba58d8dbb: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 18:48:49.183279 28259 tablet_replica.cc:439] T 2c26911d42324f4abe7b8efefd35489b P 7895952636a34ed7b8d53efba58d8dbb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } } peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } }
I20260810 18:48:49.183413 28259 tablet_replica.cc:442] T 2c26911d42324f4abe7b8efefd35489b P 7895952636a34ed7b8d53efba58d8dbb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:48:49.239624 28260 raft_consensus.cc:482] T 2c26911d42324f4abe7b8efefd35489b P 87492194659641e2a9141f7233bcc623 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:48:49.239777 28260 raft_consensus.cc:504] T 2c26911d42324f4abe7b8efefd35489b P 87492194659641e2a9141f7233bcc623 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } } peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } }
I20260810 18:48:49.240244 28260 leader_election.cc:288] T 2c26911d42324f4abe7b8efefd35489b P 87492194659641e2a9141f7233bcc623 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a07e4e0445a140de92c89b90c930e418 (127.18.168.131:43327), 7895952636a34ed7b8d53efba58d8dbb (127.18.168.130:37203)
I20260810 18:48:49.240743 28153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c26911d42324f4abe7b8efefd35489b" candidate_uuid: "87492194659641e2a9141f7233bcc623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a07e4e0445a140de92c89b90c930e418" is_pre_election: true
I20260810 18:48:49.240862 27987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c26911d42324f4abe7b8efefd35489b" candidate_uuid: "87492194659641e2a9141f7233bcc623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7895952636a34ed7b8d53efba58d8dbb" is_pre_election: true
I20260810 18:48:49.240926 28153 raft_consensus.cc:2454] T 2c26911d42324f4abe7b8efefd35489b P a07e4e0445a140de92c89b90c930e418 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87492194659641e2a9141f7233bcc623 in term 0.
I20260810 18:48:49.241039 27987 raft_consensus.cc:2454] T 2c26911d42324f4abe7b8efefd35489b P 7895952636a34ed7b8d53efba58d8dbb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87492194659641e2a9141f7233bcc623 in term 0.
I20260810 18:48:49.241289 27741 leader_election.cc:302] T 2c26911d42324f4abe7b8efefd35489b P 87492194659641e2a9141f7233bcc623 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87492194659641e2a9141f7233bcc623, a07e4e0445a140de92c89b90c930e418; no voters:
I20260810 18:48:49.241478 28260 raft_consensus.cc:2781] T 2c26911d42324f4abe7b8efefd35489b P 87492194659641e2a9141f7233bcc623 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:48:49.241542 28260 raft_consensus.cc:482] T 2c26911d42324f4abe7b8efefd35489b P 87492194659641e2a9141f7233bcc623 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:48:49.241583 28260 raft_consensus.cc:3037] T 2c26911d42324f4abe7b8efefd35489b P 87492194659641e2a9141f7233bcc623 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:49.242600 28260 raft_consensus.cc:504] T 2c26911d42324f4abe7b8efefd35489b P 87492194659641e2a9141f7233bcc623 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } } peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } }
I20260810 18:48:49.242921 28260 leader_election.cc:288] T 2c26911d42324f4abe7b8efefd35489b P 87492194659641e2a9141f7233bcc623 [CANDIDATE]: Term 1 election: Requested vote from peers a07e4e0445a140de92c89b90c930e418 (127.18.168.131:43327), 7895952636a34ed7b8d53efba58d8dbb (127.18.168.130:37203)
I20260810 18:48:49.243237 28153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c26911d42324f4abe7b8efefd35489b" candidate_uuid: "87492194659641e2a9141f7233bcc623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a07e4e0445a140de92c89b90c930e418"
I20260810 18:48:49.243353 28153 raft_consensus.cc:3037] T 2c26911d42324f4abe7b8efefd35489b P a07e4e0445a140de92c89b90c930e418 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:49.243343 27987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c26911d42324f4abe7b8efefd35489b" candidate_uuid: "87492194659641e2a9141f7233bcc623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7895952636a34ed7b8d53efba58d8dbb"
I20260810 18:48:49.243476 27987 raft_consensus.cc:3037] T 2c26911d42324f4abe7b8efefd35489b P 7895952636a34ed7b8d53efba58d8dbb [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:49.244338 28153 raft_consensus.cc:2454] T 2c26911d42324f4abe7b8efefd35489b P a07e4e0445a140de92c89b90c930e418 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87492194659641e2a9141f7233bcc623 in term 1.
I20260810 18:48:49.244642 27741 leader_election.cc:302] T 2c26911d42324f4abe7b8efefd35489b P 87492194659641e2a9141f7233bcc623 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87492194659641e2a9141f7233bcc623, a07e4e0445a140de92c89b90c930e418; no voters:
I20260810 18:48:49.244716 27987 raft_consensus.cc:2454] T 2c26911d42324f4abe7b8efefd35489b P 7895952636a34ed7b8d53efba58d8dbb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87492194659641e2a9141f7233bcc623 in term 1.
I20260810 18:48:49.244865 28260 raft_consensus.cc:2781] T 2c26911d42324f4abe7b8efefd35489b P 87492194659641e2a9141f7233bcc623 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:49.244972 28260 raft_consensus.cc:686] T 2c26911d42324f4abe7b8efefd35489b P 87492194659641e2a9141f7233bcc623 [term 1 LEADER]: Becoming Leader. State: Replica: 87492194659641e2a9141f7233bcc623, State: Running, Role: LEADER
I20260810 18:48:49.245148 28260 consensus_queue.cc:237] T 2c26911d42324f4abe7b8efefd35489b P 87492194659641e2a9141f7233bcc623 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } } peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } }
I20260810 18:48:49.245846 28260 tablet_replica.cc:439] T 2c26911d42324f4abe7b8efefd35489b P 87492194659641e2a9141f7233bcc623: TxnStatusTablet state changed. Reason: New leader 87492194659641e2a9141f7233bcc623. Latest consensus state: current_term: 1 leader_uuid: "87492194659641e2a9141f7233bcc623" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } } peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } }
I20260810 18:48:49.245946 28260 tablet_replica.cc:442] T 2c26911d42324f4abe7b8efefd35489b P 87492194659641e2a9141f7233bcc623: This TxnStatusTablet replica's current role is: LEADER
I20260810 18:48:49.246235 28301 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 18:48:49.246409 28301 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 18:48:49.247058 27641 catalog_manager.cc:5495] T 2c26911d42324f4abe7b8efefd35489b P 87492194659641e2a9141f7233bcc623 reported cstate change: term changed from 0 to 1, leader changed from <none> to 87492194659641e2a9141f7233bcc623 (127.18.168.129). New cstate: current_term: 1 leader_uuid: "87492194659641e2a9141f7233bcc623" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } health_report { overall_health: UNKNOWN } } }
I20260810 18:48:49.261475 28295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:49.257209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:43315 (local address 127.0.0.1:51334)
0810 18:48:49.257370 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:49.257387 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:49.257492 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:49.257928 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:49.257931 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:49.257938 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:48:49.258051 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:49.258056 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.259036 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.259038 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:49.259734 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:49.259743 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.260309 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.260312 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:49.260385 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:49.260430 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:48:49.261225 (+ 795us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:48:49.261240 (+ 15us) client_negotiation.cc:769] Sending connection context
0810 18:48:49.261298 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 18:48:49.261368 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260810 18:48:49.261528 28248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:49.257505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51334 (local address 127.18.168.129:43315)
0810 18:48:49.257625 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:49.257630 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:49.257646 (+ 16us) server_negotiation.cc:407] Connection header received
0810 18:48:49.257721 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:49.257724 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:49.257792 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:49.257913 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:48:49.258166 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.258915 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.259846 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.260195 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.260317 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:49.260573 (+ 256us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:48:49.261117 (+ 544us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:48:49.261248 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:49.261316 (+ 68us) server_negotiation.cc:299] Negotiation successful
0810 18:48:49.261377 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 18:48:49.264874 27987 raft_consensus.cc:1261] T 2c26911d42324f4abe7b8efefd35489b P 7895952636a34ed7b8d53efba58d8dbb [term 1 FOLLOWER]: Refusing update from remote peer 87492194659641e2a9141f7233bcc623: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:49.265071 28153 raft_consensus.cc:1261] T 2c26911d42324f4abe7b8efefd35489b P a07e4e0445a140de92c89b90c930e418 [term 1 FOLLOWER]: Refusing update from remote peer 87492194659641e2a9141f7233bcc623: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:49.265349 28260 consensus_queue.cc:1035] T 2c26911d42324f4abe7b8efefd35489b P 87492194659641e2a9141f7233bcc623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:49.265619 28288 consensus_queue.cc:1035] T 2c26911d42324f4abe7b8efefd35489b P 87492194659641e2a9141f7233bcc623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:49.267501 28259 tablet_replica.cc:439] T 2c26911d42324f4abe7b8efefd35489b P 7895952636a34ed7b8d53efba58d8dbb: TxnStatusTablet state changed. Reason: New leader 87492194659641e2a9141f7233bcc623. Latest consensus state: current_term: 1 leader_uuid: "87492194659641e2a9141f7233bcc623" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } } peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } }
I20260810 18:48:49.267627 28259 tablet_replica.cc:442] T 2c26911d42324f4abe7b8efefd35489b P 7895952636a34ed7b8d53efba58d8dbb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:48:49.268291 28258 tablet_replica.cc:439] T 2c26911d42324f4abe7b8efefd35489b P a07e4e0445a140de92c89b90c930e418: TxnStatusTablet state changed. Reason: New leader 87492194659641e2a9141f7233bcc623. Latest consensus state: current_term: 1 leader_uuid: "87492194659641e2a9141f7233bcc623" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } } peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } }
I20260810 18:48:49.268419 28258 tablet_replica.cc:442] T 2c26911d42324f4abe7b8efefd35489b P a07e4e0445a140de92c89b90c930e418: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:48:49.270218 28260 tablet_replica.cc:439] T 2c26911d42324f4abe7b8efefd35489b P 87492194659641e2a9141f7233bcc623: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "87492194659641e2a9141f7233bcc623" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } } peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } }
I20260810 18:48:49.270337 28260 tablet_replica.cc:442] T 2c26911d42324f4abe7b8efefd35489b P 87492194659641e2a9141f7233bcc623: This TxnStatusTablet replica's current role is: LEADER
I20260810 18:48:49.270524 28288 tablet_replica.cc:439] T 2c26911d42324f4abe7b8efefd35489b P 87492194659641e2a9141f7233bcc623: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "87492194659641e2a9141f7233bcc623" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } } peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } }
I20260810 18:48:49.270608 28288 tablet_replica.cc:442] T 2c26911d42324f4abe7b8efefd35489b P 87492194659641e2a9141f7233bcc623: This TxnStatusTablet replica's current role is: LEADER
I20260810 18:48:49.270618 28268 mvcc.cc:205] Tried to move back new op lower bound from 7317044139063758848 to 7317044138989346816. Current Snapshot: MvccSnapshot[applied={T|T < 7317044139063758848}]
I20260810 18:48:49.271692 28259 tablet_replica.cc:439] T 2c26911d42324f4abe7b8efefd35489b P 7895952636a34ed7b8d53efba58d8dbb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "87492194659641e2a9141f7233bcc623" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } } peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } }
I20260810 18:48:49.271818 28259 tablet_replica.cc:442] T 2c26911d42324f4abe7b8efefd35489b P 7895952636a34ed7b8d53efba58d8dbb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:48:49.272200 28269 mvcc.cc:205] Tried to move back new op lower bound from 7317044139063758848 to 7317044138989346816. Current Snapshot: MvccSnapshot[applied={T|T < 7317044139063758848 or (T in {7317044139063758848})}]
I20260810 18:48:49.272923 28267 mvcc.cc:205] Tried to move back new op lower bound from 7317044139063758848 to 7317044138989346816. Current Snapshot: MvccSnapshot[applied={T|T < 7317044139063758848}]
I20260810 18:48:49.273841 28258 tablet_replica.cc:439] T 2c26911d42324f4abe7b8efefd35489b P a07e4e0445a140de92c89b90c930e418: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "87492194659641e2a9141f7233bcc623" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } } peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } }
I20260810 18:48:49.273964 28258 tablet_replica.cc:442] T 2c26911d42324f4abe7b8efefd35489b P a07e4e0445a140de92c89b90c930e418: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:48:49.279313 28248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:49.275590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51344 (local address 127.18.168.129:43315)
0810 18:48:49.275708 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:49.275713 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:49.275727 (+ 14us) server_negotiation.cc:407] Connection header received
0810 18:48:49.275773 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:49.275777 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:49.275831 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:49.275920 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:48:49.276225 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.276911 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.278238 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.278453 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.278541 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:49.278863 (+ 322us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:48:49.278950 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:48:49.279034 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:49.279142 (+ 108us) server_negotiation.cc:299] Negotiation successful
0810 18:48:49.279185 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 18:48:49.289659 28310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:49.285952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:43315 (local address 127.18.168.129:52593)
0810 18:48:49.286310 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:49.286322 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:49.286402 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:49.286620 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:49.286622 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:49.286627 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:48:49.286745 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:49.286749 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.287755 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.287757 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:49.288338 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:49.288345 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.288968 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.288970 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:49.289083 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:49.289124 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:48:49.289411 (+ 287us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:48:49.289415 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 18:48:49.289462 (+ 47us) client_negotiation.cc:240] Negotiation successful
0810 18:48:49.289518 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":32,"thread_start_us":107,"threads_started":1}
I20260810 18:48:49.289670 28248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:49.286064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:52593 (local address 127.18.168.129:43315)
0810 18:48:49.286190 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:49.286194 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:49.286387 (+ 193us) server_negotiation.cc:407] Connection header received
0810 18:48:49.286469 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:49.286472 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:49.286522 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:49.286604 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:48:49.286845 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.287659 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.288445 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.288862 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.289025 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:49.289249 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:48:49.289324 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:48:49.289401 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:49.289476 (+ 75us) server_negotiation.cc:299] Negotiation successful
0810 18:48:49.289525 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":47}
I20260810 18:48:49.296269 27987 raft_consensus.cc:1261] T 7fd2b1de41094b10bff247a31a19f6c6 P 7895952636a34ed7b8d53efba58d8dbb [term 1 FOLLOWER]: Refusing update from remote peer 87492194659641e2a9141f7233bcc623: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:49.296387 28153 raft_consensus.cc:1261] T 7fd2b1de41094b10bff247a31a19f6c6 P a07e4e0445a140de92c89b90c930e418 [term 1 FOLLOWER]: Refusing update from remote peer 87492194659641e2a9141f7233bcc623: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:49.296661 28260 consensus_queue.cc:1035] T 7fd2b1de41094b10bff247a31a19f6c6 P 87492194659641e2a9141f7233bcc623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:49.296830 28307 consensus_queue.cc:1035] T 7fd2b1de41094b10bff247a31a19f6c6 P 87492194659641e2a9141f7233bcc623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:49.350098 27643 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:49.350245 27643 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:49.352334 27643 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dfbc86f800924b13a83b5a0e342be613: Sending DeleteTablet for 3 replicas of tablet 7fd2b1de41094b10bff247a31a19f6c6
I20260810 18:48:49.352960 28132 tablet_service.cc:1514] Processing DeleteTablet for tablet 7fd2b1de41094b10bff247a31a19f6c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:49 UTC) from {username='oryx', principal='oryx/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:39218
I20260810 18:48:49.352946 27963 tablet_service.cc:1514] Processing DeleteTablet for tablet 7fd2b1de41094b10bff247a31a19f6c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:49 UTC) from {username='oryx', principal='oryx/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:39090
I20260810 18:48:49.353132 28281 tablet_replica.cc:330] T 7fd2b1de41094b10bff247a31a19f6c6 P 7895952636a34ed7b8d53efba58d8dbb: stopping tablet replica
I20260810 18:48:49.353132 28280 tablet_replica.cc:330] T 7fd2b1de41094b10bff247a31a19f6c6 P a07e4e0445a140de92c89b90c930e418: stopping tablet replica
I20260810 18:48:49.353242 28281 raft_consensus.cc:2229] T 7fd2b1de41094b10bff247a31a19f6c6 P 7895952636a34ed7b8d53efba58d8dbb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:49.353242 28280 raft_consensus.cc:2229] T 7fd2b1de41094b10bff247a31a19f6c6 P a07e4e0445a140de92c89b90c930e418 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:49.353355 28280 raft_consensus.cc:2258] T 7fd2b1de41094b10bff247a31a19f6c6 P a07e4e0445a140de92c89b90c930e418 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:49.353355 28281 raft_consensus.cc:2258] T 7fd2b1de41094b10bff247a31a19f6c6 P 7895952636a34ed7b8d53efba58d8dbb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:49.353547 27799 tablet_service.cc:1514] Processing DeleteTablet for tablet 7fd2b1de41094b10bff247a31a19f6c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:49 UTC) from {username='oryx', principal='oryx/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:51332
I20260810 18:48:49.353722 28279 tablet_replica.cc:330] T 7fd2b1de41094b10bff247a31a19f6c6 P 87492194659641e2a9141f7233bcc623: stopping tablet replica
I20260810 18:48:49.353839 28279 raft_consensus.cc:2229] T 7fd2b1de41094b10bff247a31a19f6c6 P 87492194659641e2a9141f7233bcc623 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:48:49.354070 28280 ts_tablet_manager.cc:1905] T 7fd2b1de41094b10bff247a31a19f6c6 P a07e4e0445a140de92c89b90c930e418: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:49.354135 28279 raft_consensus.cc:2258] T 7fd2b1de41094b10bff247a31a19f6c6 P 87492194659641e2a9141f7233bcc623 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:49.354352 28281 ts_tablet_manager.cc:1905] T 7fd2b1de41094b10bff247a31a19f6c6 P 7895952636a34ed7b8d53efba58d8dbb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:49.355083 28279 ts_tablet_manager.cc:1905] T 7fd2b1de41094b10bff247a31a19f6c6 P 87492194659641e2a9141f7233bcc623: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:49.357215 28280 ts_tablet_manager.cc:1918] T 7fd2b1de41094b10bff247a31a19f6c6 P a07e4e0445a140de92c89b90c930e418: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:48:49.357295 28280 log.cc:1196] T 7fd2b1de41094b10bff247a31a19f6c6 P a07e4e0445a140de92c89b90c930e418: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-2/wal/wals/7fd2b1de41094b10bff247a31a19f6c6
I20260810 18:48:49.357288 28281 ts_tablet_manager.cc:1918] T 7fd2b1de41094b10bff247a31a19f6c6 P 7895952636a34ed7b8d53efba58d8dbb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:48:49.357367 28281 log.cc:1196] T 7fd2b1de41094b10bff247a31a19f6c6 P 7895952636a34ed7b8d53efba58d8dbb: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-1/wal/wals/7fd2b1de41094b10bff247a31a19f6c6
I20260810 18:48:49.357569 28280 ts_tablet_manager.cc:1939] T 7fd2b1de41094b10bff247a31a19f6c6 P a07e4e0445a140de92c89b90c930e418: Deleting consensus metadata
I20260810 18:48:49.357656 28281 ts_tablet_manager.cc:1939] T 7fd2b1de41094b10bff247a31a19f6c6 P 7895952636a34ed7b8d53efba58d8dbb: Deleting consensus metadata
I20260810 18:48:49.358386 27625 catalog_manager.cc:4841] TS a07e4e0445a140de92c89b90c930e418 (127.18.168.131:43327): tablet 7fd2b1de41094b10bff247a31a19f6c6 (table test-table [id=2670a64ab2ed4c0ab8e9494daa4a5344]) successfully deleted
I20260810 18:48:49.358498 27627 catalog_manager.cc:4841] TS 7895952636a34ed7b8d53efba58d8dbb (127.18.168.130:37203): tablet 7fd2b1de41094b10bff247a31a19f6c6 (table test-table [id=2670a64ab2ed4c0ab8e9494daa4a5344]) successfully deleted
I20260810 18:48:49.362119 28279 ts_tablet_manager.cc:1918] T 7fd2b1de41094b10bff247a31a19f6c6 P 87492194659641e2a9141f7233bcc623: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:48:49.362212 28279 log.cc:1196] T 7fd2b1de41094b10bff247a31a19f6c6 P 87492194659641e2a9141f7233bcc623: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-0/wal/wals/7fd2b1de41094b10bff247a31a19f6c6
I20260810 18:48:49.362489 28279 ts_tablet_manager.cc:1939] T 7fd2b1de41094b10bff247a31a19f6c6 P 87492194659641e2a9141f7233bcc623: Deleting consensus metadata
I20260810 18:48:49.363404 27625 catalog_manager.cc:4841] TS 87492194659641e2a9141f7233bcc623 (127.18.168.129:43315): tablet 7fd2b1de41094b10bff247a31a19f6c6 (table test-table [id=2670a64ab2ed4c0ab8e9494daa4a5344]) successfully deleted
Aug 10 18:48:49 dist-test-slave-cbsf krb5kdc[27593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387729, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 18:48:49 dist-test-slave-cbsf krb5kdc[27593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:48:49 dist-test-slave-cbsf krb5kdc[27593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
W20260810 18:48:49.387403 27700 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:49.376967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52594 (local address 127.18.168.190:43829)
0810 18:48:49.377131 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:49.377137 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:49.377348 (+ 211us) server_negotiation.cc:407] Connection header received
0810 18:48:49.377507 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:49.377510 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:49.377559 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:49.377631 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:49.378623 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.379165 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.380476 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.380683 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.380807 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:49.387264 (+ 6457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52594: BlockingRecv error: recv got EOF from 127.0.0.1:52594 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 10 18:48:49 dist-test-slave-cbsf krb5kdc[27593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387729, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM
I20260810 18:48:49.398259 27700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:49.389175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52602 (local address 127.18.168.190:43829)
0810 18:48:49.389303 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:49.389308 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:49.389459 (+ 151us) server_negotiation.cc:407] Connection header received
0810 18:48:49.389577 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:49.389578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:49.389613 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:49.389676 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:49.390416 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.390876 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.392043 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.392234 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.392305 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:49.395170 (+ 2865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:49.395173 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:49.395175 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:49.395197 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:49.396594 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:49.397036 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:49.397045 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:49.397047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:49.397121 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:49.397417 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:49.397420 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:49.397422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:49.397596 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:49.397734 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:49.397972 (+ 238us) server_negotiation.cc:299] Negotiation successful
0810 18:48:49.398093 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 18:48:49.401100 27643 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52602:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:48:49.401561 27643 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:48:49.406749 27799 tablet_service.cc:1467] Processing CreateTablet for tablet faa75d00364e43009a030a2811ac32ef (DEFAULT_TABLE table=test-table [id=dfe081514f09494aabc6e58b5a0f8c78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:49.406950 28132 tablet_service.cc:1467] Processing CreateTablet for tablet faa75d00364e43009a030a2811ac32ef (DEFAULT_TABLE table=test-table [id=dfe081514f09494aabc6e58b5a0f8c78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:49.407079 27799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faa75d00364e43009a030a2811ac32ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:49.407028 27963 tablet_service.cc:1467] Processing CreateTablet for tablet faa75d00364e43009a030a2811ac32ef (DEFAULT_TABLE table=test-table [id=dfe081514f09494aabc6e58b5a0f8c78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:49.407230 28132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faa75d00364e43009a030a2811ac32ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:49.407341 27963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faa75d00364e43009a030a2811ac32ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:49.409694 28253 tablet_bootstrap.cc:492] T faa75d00364e43009a030a2811ac32ef P a07e4e0445a140de92c89b90c930e418: Bootstrap starting.
I20260810 18:48:49.409766 28250 tablet_bootstrap.cc:492] T faa75d00364e43009a030a2811ac32ef P 87492194659641e2a9141f7233bcc623: Bootstrap starting.
I20260810 18:48:49.409989 28249 tablet_bootstrap.cc:492] T faa75d00364e43009a030a2811ac32ef P 7895952636a34ed7b8d53efba58d8dbb: Bootstrap starting.
I20260810 18:48:49.410635 28253 tablet_bootstrap.cc:654] T faa75d00364e43009a030a2811ac32ef P a07e4e0445a140de92c89b90c930e418: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:49.410635 28250 tablet_bootstrap.cc:654] T faa75d00364e43009a030a2811ac32ef P 87492194659641e2a9141f7233bcc623: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:49.411000 28249 tablet_bootstrap.cc:654] T faa75d00364e43009a030a2811ac32ef P 7895952636a34ed7b8d53efba58d8dbb: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:49.412091 28253 tablet_bootstrap.cc:492] T faa75d00364e43009a030a2811ac32ef P a07e4e0445a140de92c89b90c930e418: No bootstrap required, opened a new log
I20260810 18:48:49.412194 28253 ts_tablet_manager.cc:1397] T faa75d00364e43009a030a2811ac32ef P a07e4e0445a140de92c89b90c930e418: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 18:48:49.412611 28249 tablet_bootstrap.cc:492] T faa75d00364e43009a030a2811ac32ef P 7895952636a34ed7b8d53efba58d8dbb: No bootstrap required, opened a new log
I20260810 18:48:49.412734 28249 ts_tablet_manager.cc:1397] T faa75d00364e43009a030a2811ac32ef P 7895952636a34ed7b8d53efba58d8dbb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 18:48:49.412744 28250 tablet_bootstrap.cc:492] T faa75d00364e43009a030a2811ac32ef P 87492194659641e2a9141f7233bcc623: No bootstrap required, opened a new log
I20260810 18:48:49.412657 28253 raft_consensus.cc:348] T faa75d00364e43009a030a2811ac32ef P a07e4e0445a140de92c89b90c930e418 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } }
I20260810 18:48:49.412789 28253 raft_consensus.cc:374] T faa75d00364e43009a030a2811ac32ef P a07e4e0445a140de92c89b90c930e418 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:49.412822 28250 ts_tablet_manager.cc:1397] T faa75d00364e43009a030a2811ac32ef P 87492194659641e2a9141f7233bcc623: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260810 18:48:49.412822 28253 raft_consensus.cc:729] T faa75d00364e43009a030a2811ac32ef P a07e4e0445a140de92c89b90c930e418 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a07e4e0445a140de92c89b90c930e418, State: Initialized, Role: FOLLOWER
I20260810 18:48:49.412954 28253 consensus_queue.cc:260] T faa75d00364e43009a030a2811ac32ef P a07e4e0445a140de92c89b90c930e418 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } }
I20260810 18:48:49.413218 28253 ts_tablet_manager.cc:1428] T faa75d00364e43009a030a2811ac32ef P a07e4e0445a140de92c89b90c930e418: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 18:48:49.413209 28249 raft_consensus.cc:348] T faa75d00364e43009a030a2811ac32ef P 7895952636a34ed7b8d53efba58d8dbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } }
I20260810 18:48:49.413254 28250 raft_consensus.cc:348] T faa75d00364e43009a030a2811ac32ef P 87492194659641e2a9141f7233bcc623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } }
I20260810 18:48:49.413316 28249 raft_consensus.cc:374] T faa75d00364e43009a030a2811ac32ef P 7895952636a34ed7b8d53efba58d8dbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:49.413354 28249 raft_consensus.cc:729] T faa75d00364e43009a030a2811ac32ef P 7895952636a34ed7b8d53efba58d8dbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7895952636a34ed7b8d53efba58d8dbb, State: Initialized, Role: FOLLOWER
I20260810 18:48:49.413360 28250 raft_consensus.cc:374] T faa75d00364e43009a030a2811ac32ef P 87492194659641e2a9141f7233bcc623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:49.413393 28250 raft_consensus.cc:729] T faa75d00364e43009a030a2811ac32ef P 87492194659641e2a9141f7233bcc623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87492194659641e2a9141f7233bcc623, State: Initialized, Role: FOLLOWER
I20260810 18:48:49.413480 28249 consensus_queue.cc:260] T faa75d00364e43009a030a2811ac32ef P 7895952636a34ed7b8d53efba58d8dbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } }
I20260810 18:48:49.413504 28250 consensus_queue.cc:260] T faa75d00364e43009a030a2811ac32ef P 87492194659641e2a9141f7233bcc623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } }
I20260810 18:48:49.413754 28249 ts_tablet_manager.cc:1428] T faa75d00364e43009a030a2811ac32ef P 7895952636a34ed7b8d53efba58d8dbb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 18:48:49.413767 28250 ts_tablet_manager.cc:1428] T faa75d00364e43009a030a2811ac32ef P 87492194659641e2a9141f7233bcc623: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 18:48:49.540334 28258 raft_consensus.cc:482] T faa75d00364e43009a030a2811ac32ef P a07e4e0445a140de92c89b90c930e418 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:48:49.540514 28258 raft_consensus.cc:504] T faa75d00364e43009a030a2811ac32ef P a07e4e0445a140de92c89b90c930e418 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } }
I20260810 18:48:49.540998 28258 leader_election.cc:288] T faa75d00364e43009a030a2811ac32ef P a07e4e0445a140de92c89b90c930e418 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87492194659641e2a9141f7233bcc623 (127.18.168.129:43315), 7895952636a34ed7b8d53efba58d8dbb (127.18.168.130:37203)
I20260810 18:48:49.541426 27823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faa75d00364e43009a030a2811ac32ef" candidate_uuid: "a07e4e0445a140de92c89b90c930e418" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87492194659641e2a9141f7233bcc623" is_pre_election: true
I20260810 18:48:49.541599 27823 raft_consensus.cc:2454] T faa75d00364e43009a030a2811ac32ef P 87492194659641e2a9141f7233bcc623 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a07e4e0445a140de92c89b90c930e418 in term 0.
I20260810 18:48:49.541764 27986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faa75d00364e43009a030a2811ac32ef" candidate_uuid: "a07e4e0445a140de92c89b90c930e418" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7895952636a34ed7b8d53efba58d8dbb" is_pre_election: true
I20260810 18:48:49.541920 27986 raft_consensus.cc:2454] T faa75d00364e43009a030a2811ac32ef P 7895952636a34ed7b8d53efba58d8dbb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a07e4e0445a140de92c89b90c930e418 in term 0.
I20260810 18:48:49.541952 28071 leader_election.cc:302] T faa75d00364e43009a030a2811ac32ef P a07e4e0445a140de92c89b90c930e418 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87492194659641e2a9141f7233bcc623, a07e4e0445a140de92c89b90c930e418; no voters:
I20260810 18:48:49.542119 28258 raft_consensus.cc:2781] T faa75d00364e43009a030a2811ac32ef P a07e4e0445a140de92c89b90c930e418 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:48:49.542162 28258 raft_consensus.cc:482] T faa75d00364e43009a030a2811ac32ef P a07e4e0445a140de92c89b90c930e418 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:48:49.542196 28258 raft_consensus.cc:3037] T faa75d00364e43009a030a2811ac32ef P a07e4e0445a140de92c89b90c930e418 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:49.543460 28258 raft_consensus.cc:504] T faa75d00364e43009a030a2811ac32ef P a07e4e0445a140de92c89b90c930e418 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } }
I20260810 18:48:49.543781 28258 leader_election.cc:288] T faa75d00364e43009a030a2811ac32ef P a07e4e0445a140de92c89b90c930e418 [CANDIDATE]: Term 1 election: Requested vote from peers 87492194659641e2a9141f7233bcc623 (127.18.168.129:43315), 7895952636a34ed7b8d53efba58d8dbb (127.18.168.130:37203)
I20260810 18:48:49.544180 27823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faa75d00364e43009a030a2811ac32ef" candidate_uuid: "a07e4e0445a140de92c89b90c930e418" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87492194659641e2a9141f7233bcc623"
I20260810 18:48:49.544205 27986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faa75d00364e43009a030a2811ac32ef" candidate_uuid: "a07e4e0445a140de92c89b90c930e418" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7895952636a34ed7b8d53efba58d8dbb"
I20260810 18:48:49.544286 27823 raft_consensus.cc:3037] T faa75d00364e43009a030a2811ac32ef P 87492194659641e2a9141f7233bcc623 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:49.544323 27986 raft_consensus.cc:3037] T faa75d00364e43009a030a2811ac32ef P 7895952636a34ed7b8d53efba58d8dbb [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:49.545452 27823 raft_consensus.cc:2454] T faa75d00364e43009a030a2811ac32ef P 87492194659641e2a9141f7233bcc623 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a07e4e0445a140de92c89b90c930e418 in term 1.
I20260810 18:48:49.545550 27986 raft_consensus.cc:2454] T faa75d00364e43009a030a2811ac32ef P 7895952636a34ed7b8d53efba58d8dbb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a07e4e0445a140de92c89b90c930e418 in term 1.
I20260810 18:48:49.545730 28071 leader_election.cc:302] T faa75d00364e43009a030a2811ac32ef P a07e4e0445a140de92c89b90c930e418 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87492194659641e2a9141f7233bcc623, a07e4e0445a140de92c89b90c930e418; no voters:
I20260810 18:48:49.545944 28258 raft_consensus.cc:2781] T faa75d00364e43009a030a2811ac32ef P a07e4e0445a140de92c89b90c930e418 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:49.546083 28258 raft_consensus.cc:686] T faa75d00364e43009a030a2811ac32ef P a07e4e0445a140de92c89b90c930e418 [term 1 LEADER]: Becoming Leader. State: Replica: a07e4e0445a140de92c89b90c930e418, State: Running, Role: LEADER
I20260810 18:48:49.546227 28258 consensus_queue.cc:237] T faa75d00364e43009a030a2811ac32ef P a07e4e0445a140de92c89b90c930e418 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } }
I20260810 18:48:49.548069 27642 catalog_manager.cc:5495] T faa75d00364e43009a030a2811ac32ef P a07e4e0445a140de92c89b90c930e418 reported cstate change: term changed from 0 to 1, leader changed from <none> to a07e4e0445a140de92c89b90c930e418 (127.18.168.131). New cstate: current_term: 1 leader_uuid: "a07e4e0445a140de92c89b90c930e418" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } health_report { overall_health: HEALTHY } } }
I20260810 18:48:49.572603 28246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:49.568965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39240 (local address 127.18.168.131:43327)
0810 18:48:49.569115 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:49.569119 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:49.569133 (+ 14us) server_negotiation.cc:407] Connection header received
0810 18:48:49.569246 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:49.569249 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:49.569293 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:49.569355 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:48:49.569658 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.570164 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.571411 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.571631 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.571711 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:49.572086 (+ 375us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:48:49.572160 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:48:49.572232 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:49.572390 (+ 158us) server_negotiation.cc:299] Negotiation successful
0810 18:48:49.572442 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 18:48:49.574586 27823 raft_consensus.cc:1261] T faa75d00364e43009a030a2811ac32ef P 87492194659641e2a9141f7233bcc623 [term 1 FOLLOWER]: Refusing update from remote peer a07e4e0445a140de92c89b90c930e418: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:49.574607 27986 raft_consensus.cc:1261] T faa75d00364e43009a030a2811ac32ef P 7895952636a34ed7b8d53efba58d8dbb [term 1 FOLLOWER]: Refusing update from remote peer a07e4e0445a140de92c89b90c930e418: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:49.575044 28258 consensus_queue.cc:1035] T faa75d00364e43009a030a2811ac32ef P a07e4e0445a140de92c89b90c930e418 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:49.575208 28333 consensus_queue.cc:1035] T faa75d00364e43009a030a2811ac32ef P a07e4e0445a140de92c89b90c930e418 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:49.582056 27641 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52602:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:49.582195 27641 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52602:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:49.585201 27641 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dfbc86f800924b13a83b5a0e342be613: Sending DeleteTablet for 3 replicas of tablet faa75d00364e43009a030a2811ac32ef
I20260810 18:48:49.586122 27799 tablet_service.cc:1514] Processing DeleteTablet for tablet faa75d00364e43009a030a2811ac32ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:49 UTC) from {username='oryx', principal='oryx/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:51332
I20260810 18:48:49.586167 27963 tablet_service.cc:1514] Processing DeleteTablet for tablet faa75d00364e43009a030a2811ac32ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:49 UTC) from {username='oryx', principal='oryx/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:39090
I20260810 18:48:49.586323 28132 tablet_service.cc:1514] Processing DeleteTablet for tablet faa75d00364e43009a030a2811ac32ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:49 UTC) from {username='oryx', principal='oryx/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:39218
I20260810 18:48:49.586355 27962 tablet_service.cc:1514] Processing DeleteTablet for tablet faa75d00364e43009a030a2811ac32ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:49 UTC) from {username='oryx', principal='oryx/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:39090
I20260810 18:48:49.586484 28280 tablet_replica.cc:330] T faa75d00364e43009a030a2811ac32ef P a07e4e0445a140de92c89b90c930e418: stopping tablet replica
I20260810 18:48:49.586495 28281 tablet_replica.cc:330] T faa75d00364e43009a030a2811ac32ef P 7895952636a34ed7b8d53efba58d8dbb: stopping tablet replica
I20260810 18:48:49.586504 28279 tablet_replica.cc:330] T faa75d00364e43009a030a2811ac32ef P 87492194659641e2a9141f7233bcc623: stopping tablet replica
I20260810 18:48:49.586602 28280 raft_consensus.cc:2229] T faa75d00364e43009a030a2811ac32ef P a07e4e0445a140de92c89b90c930e418 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:48:49.586607 28279 raft_consensus.cc:2229] T faa75d00364e43009a030a2811ac32ef P 87492194659641e2a9141f7233bcc623 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:49.586604 28281 raft_consensus.cc:2229] T faa75d00364e43009a030a2811ac32ef P 7895952636a34ed7b8d53efba58d8dbb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:49.586716 28279 raft_consensus.cc:2258] T faa75d00364e43009a030a2811ac32ef P 87492194659641e2a9141f7233bcc623 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:49.586730 28281 raft_consensus.cc:2258] T faa75d00364e43009a030a2811ac32ef P 7895952636a34ed7b8d53efba58d8dbb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:49.586779 28280 raft_consensus.cc:2258] T faa75d00364e43009a030a2811ac32ef P a07e4e0445a140de92c89b90c930e418 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:49.587317 27641 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52602:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 18:48:49.587513 28280 ts_tablet_manager.cc:1905] T faa75d00364e43009a030a2811ac32ef P a07e4e0445a140de92c89b90c930e418: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 18:48:49.587731 27641 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:48:49.588866 28279 ts_tablet_manager.cc:1905] T faa75d00364e43009a030a2811ac32ef P 87492194659641e2a9141f7233bcc623: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:49.589071 28281 ts_tablet_manager.cc:1905] T faa75d00364e43009a030a2811ac32ef P 7895952636a34ed7b8d53efba58d8dbb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:49.590083 28280 ts_tablet_manager.cc:1918] T faa75d00364e43009a030a2811ac32ef P a07e4e0445a140de92c89b90c930e418: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:48:49.590166 28280 log.cc:1196] T faa75d00364e43009a030a2811ac32ef P a07e4e0445a140de92c89b90c930e418: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-2/wal/wals/faa75d00364e43009a030a2811ac32ef
I20260810 18:48:49.590435 28280 ts_tablet_manager.cc:1939] T faa75d00364e43009a030a2811ac32ef P a07e4e0445a140de92c89b90c930e418: Deleting consensus metadata
I20260810 18:48:49.591068 27625 catalog_manager.cc:4841] TS a07e4e0445a140de92c89b90c930e418 (127.18.168.131:43327): tablet faa75d00364e43009a030a2811ac32ef (table test-table [id=dfe081514f09494aabc6e58b5a0f8c78]) successfully deleted
I20260810 18:48:49.591387 28279 ts_tablet_manager.cc:1918] T faa75d00364e43009a030a2811ac32ef P 87492194659641e2a9141f7233bcc623: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:48:49.591450 28279 log.cc:1196] T faa75d00364e43009a030a2811ac32ef P 87492194659641e2a9141f7233bcc623: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-0/wal/wals/faa75d00364e43009a030a2811ac32ef
I20260810 18:48:49.591542 28281 ts_tablet_manager.cc:1918] T faa75d00364e43009a030a2811ac32ef P 7895952636a34ed7b8d53efba58d8dbb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:48:49.591609 28281 log.cc:1196] T faa75d00364e43009a030a2811ac32ef P 7895952636a34ed7b8d53efba58d8dbb: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-1/wal/wals/faa75d00364e43009a030a2811ac32ef
I20260810 18:48:49.591631 28279 ts_tablet_manager.cc:1939] T faa75d00364e43009a030a2811ac32ef P 87492194659641e2a9141f7233bcc623: Deleting consensus metadata
I20260810 18:48:49.591847 28281 ts_tablet_manager.cc:1939] T faa75d00364e43009a030a2811ac32ef P 7895952636a34ed7b8d53efba58d8dbb: Deleting consensus metadata
I20260810 18:48:49.592162 27625 catalog_manager.cc:4841] TS 87492194659641e2a9141f7233bcc623 (127.18.168.129:43315): tablet faa75d00364e43009a030a2811ac32ef (table test-table [id=dfe081514f09494aabc6e58b5a0f8c78]) successfully deleted
I20260810 18:48:49.592657 27627 catalog_manager.cc:4841] TS 7895952636a34ed7b8d53efba58d8dbb (127.18.168.130:37203): tablet faa75d00364e43009a030a2811ac32ef (table test-table [id=dfe081514f09494aabc6e58b5a0f8c78]) successfully deleted
W20260810 18:48:49.593513 27627 catalog_manager.cc:4808] TS 7895952636a34ed7b8d53efba58d8dbb (127.18.168.130:37203): delete failed for tablet faa75d00364e43009a030a2811ac32ef because the tablet was not found. No further retry: Not found: Tablet not found: faa75d00364e43009a030a2811ac32ef
I20260810 18:48:49.596755 27962 tablet_service.cc:1467] Processing CreateTablet for tablet 8da35f870a754219b104dce3b8d87846 (DEFAULT_TABLE table=test-table [id=1b7fc65a81924aecbd0ad2b7a4ae0642]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:49.596936 28132 tablet_service.cc:1467] Processing CreateTablet for tablet 8da35f870a754219b104dce3b8d87846 (DEFAULT_TABLE table=test-table [id=1b7fc65a81924aecbd0ad2b7a4ae0642]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:49.596915 27799 tablet_service.cc:1467] Processing CreateTablet for tablet 8da35f870a754219b104dce3b8d87846 (DEFAULT_TABLE table=test-table [id=1b7fc65a81924aecbd0ad2b7a4ae0642]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:49.597013 27962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8da35f870a754219b104dce3b8d87846. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:49.597149 28132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8da35f870a754219b104dce3b8d87846. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:49.597149 27799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8da35f870a754219b104dce3b8d87846. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:49.599193 28249 tablet_bootstrap.cc:492] T 8da35f870a754219b104dce3b8d87846 P 7895952636a34ed7b8d53efba58d8dbb: Bootstrap starting.
I20260810 18:48:49.599303 28253 tablet_bootstrap.cc:492] T 8da35f870a754219b104dce3b8d87846 P a07e4e0445a140de92c89b90c930e418: Bootstrap starting.
I20260810 18:48:49.599303 28250 tablet_bootstrap.cc:492] T 8da35f870a754219b104dce3b8d87846 P 87492194659641e2a9141f7233bcc623: Bootstrap starting.
I20260810 18:48:49.600071 28253 tablet_bootstrap.cc:654] T 8da35f870a754219b104dce3b8d87846 P a07e4e0445a140de92c89b90c930e418: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:49.600071 28250 tablet_bootstrap.cc:654] T 8da35f870a754219b104dce3b8d87846 P 87492194659641e2a9141f7233bcc623: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:49.600090 28249 tablet_bootstrap.cc:654] T 8da35f870a754219b104dce3b8d87846 P 7895952636a34ed7b8d53efba58d8dbb: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:49.601560 28249 tablet_bootstrap.cc:492] T 8da35f870a754219b104dce3b8d87846 P 7895952636a34ed7b8d53efba58d8dbb: No bootstrap required, opened a new log
I20260810 18:48:49.601577 28253 tablet_bootstrap.cc:492] T 8da35f870a754219b104dce3b8d87846 P a07e4e0445a140de92c89b90c930e418: No bootstrap required, opened a new log
I20260810 18:48:49.601647 28249 ts_tablet_manager.cc:1397] T 8da35f870a754219b104dce3b8d87846 P 7895952636a34ed7b8d53efba58d8dbb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 18:48:49.601661 28253 ts_tablet_manager.cc:1397] T 8da35f870a754219b104dce3b8d87846 P a07e4e0445a140de92c89b90c930e418: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 18:48:49.601783 28250 tablet_bootstrap.cc:492] T 8da35f870a754219b104dce3b8d87846 P 87492194659641e2a9141f7233bcc623: No bootstrap required, opened a new log
I20260810 18:48:49.601848 28250 ts_tablet_manager.cc:1397] T 8da35f870a754219b104dce3b8d87846 P 87492194659641e2a9141f7233bcc623: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260810 18:48:49.602042 28253 raft_consensus.cc:348] T 8da35f870a754219b104dce3b8d87846 P a07e4e0445a140de92c89b90c930e418 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } }
I20260810 18:48:49.602144 28253 raft_consensus.cc:374] T 8da35f870a754219b104dce3b8d87846 P a07e4e0445a140de92c89b90c930e418 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:49.602065 28249 raft_consensus.cc:348] T 8da35f870a754219b104dce3b8d87846 P 7895952636a34ed7b8d53efba58d8dbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } }
I20260810 18:48:49.602177 28253 raft_consensus.cc:729] T 8da35f870a754219b104dce3b8d87846 P a07e4e0445a140de92c89b90c930e418 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a07e4e0445a140de92c89b90c930e418, State: Initialized, Role: FOLLOWER
I20260810 18:48:49.602187 28249 raft_consensus.cc:374] T 8da35f870a754219b104dce3b8d87846 P 7895952636a34ed7b8d53efba58d8dbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:49.602226 28249 raft_consensus.cc:729] T 8da35f870a754219b104dce3b8d87846 P 7895952636a34ed7b8d53efba58d8dbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7895952636a34ed7b8d53efba58d8dbb, State: Initialized, Role: FOLLOWER
I20260810 18:48:49.602244 28250 raft_consensus.cc:348] T 8da35f870a754219b104dce3b8d87846 P 87492194659641e2a9141f7233bcc623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } }
I20260810 18:48:49.602340 28250 raft_consensus.cc:374] T 8da35f870a754219b104dce3b8d87846 P 87492194659641e2a9141f7233bcc623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:49.602298 28253 consensus_queue.cc:260] T 8da35f870a754219b104dce3b8d87846 P a07e4e0445a140de92c89b90c930e418 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } }
I20260810 18:48:49.602375 28250 raft_consensus.cc:729] T 8da35f870a754219b104dce3b8d87846 P 87492194659641e2a9141f7233bcc623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87492194659641e2a9141f7233bcc623, State: Initialized, Role: FOLLOWER
I20260810 18:48:49.602365 28249 consensus_queue.cc:260] T 8da35f870a754219b104dce3b8d87846 P 7895952636a34ed7b8d53efba58d8dbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } }
I20260810 18:48:49.602500 28250 consensus_queue.cc:260] T 8da35f870a754219b104dce3b8d87846 P 87492194659641e2a9141f7233bcc623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } }
I20260810 18:48:49.602620 28253 ts_tablet_manager.cc:1428] T 8da35f870a754219b104dce3b8d87846 P a07e4e0445a140de92c89b90c930e418: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 18:48:49.602663 28249 ts_tablet_manager.cc:1428] T 8da35f870a754219b104dce3b8d87846 P 7895952636a34ed7b8d53efba58d8dbb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 18:48:49.602774 28250 ts_tablet_manager.cc:1428] T 8da35f870a754219b104dce3b8d87846 P 87492194659641e2a9141f7233bcc623: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 18:48:49.657248 28307 raft_consensus.cc:482] T 8da35f870a754219b104dce3b8d87846 P 87492194659641e2a9141f7233bcc623 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:48:49.657382 28307 raft_consensus.cc:504] T 8da35f870a754219b104dce3b8d87846 P 87492194659641e2a9141f7233bcc623 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } }
I20260810 18:48:49.657850 28307 leader_election.cc:288] T 8da35f870a754219b104dce3b8d87846 P 87492194659641e2a9141f7233bcc623 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7895952636a34ed7b8d53efba58d8dbb (127.18.168.130:37203), a07e4e0445a140de92c89b90c930e418 (127.18.168.131:43327)
I20260810 18:48:49.658315 27986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8da35f870a754219b104dce3b8d87846" candidate_uuid: "87492194659641e2a9141f7233bcc623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7895952636a34ed7b8d53efba58d8dbb" is_pre_election: true
I20260810 18:48:49.658314 28153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8da35f870a754219b104dce3b8d87846" candidate_uuid: "87492194659641e2a9141f7233bcc623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a07e4e0445a140de92c89b90c930e418" is_pre_election: true
I20260810 18:48:49.658470 27986 raft_consensus.cc:2454] T 8da35f870a754219b104dce3b8d87846 P 7895952636a34ed7b8d53efba58d8dbb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87492194659641e2a9141f7233bcc623 in term 0.
I20260810 18:48:49.658470 28153 raft_consensus.cc:2454] T 8da35f870a754219b104dce3b8d87846 P a07e4e0445a140de92c89b90c930e418 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87492194659641e2a9141f7233bcc623 in term 0.
I20260810 18:48:49.658789 27744 leader_election.cc:302] T 8da35f870a754219b104dce3b8d87846 P 87492194659641e2a9141f7233bcc623 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7895952636a34ed7b8d53efba58d8dbb, 87492194659641e2a9141f7233bcc623; no voters:
I20260810 18:48:49.659041 28307 raft_consensus.cc:2781] T 8da35f870a754219b104dce3b8d87846 P 87492194659641e2a9141f7233bcc623 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:48:49.659098 28307 raft_consensus.cc:482] T 8da35f870a754219b104dce3b8d87846 P 87492194659641e2a9141f7233bcc623 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:48:49.659125 28307 raft_consensus.cc:3037] T 8da35f870a754219b104dce3b8d87846 P 87492194659641e2a9141f7233bcc623 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:49.660238 28307 raft_consensus.cc:504] T 8da35f870a754219b104dce3b8d87846 P 87492194659641e2a9141f7233bcc623 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } }
I20260810 18:48:49.660599 28307 leader_election.cc:288] T 8da35f870a754219b104dce3b8d87846 P 87492194659641e2a9141f7233bcc623 [CANDIDATE]: Term 1 election: Requested vote from peers 7895952636a34ed7b8d53efba58d8dbb (127.18.168.130:37203), a07e4e0445a140de92c89b90c930e418 (127.18.168.131:43327)
I20260810 18:48:49.660987 27986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8da35f870a754219b104dce3b8d87846" candidate_uuid: "87492194659641e2a9141f7233bcc623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7895952636a34ed7b8d53efba58d8dbb"
I20260810 18:48:49.661011 28153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8da35f870a754219b104dce3b8d87846" candidate_uuid: "87492194659641e2a9141f7233bcc623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a07e4e0445a140de92c89b90c930e418"
I20260810 18:48:49.661123 28153 raft_consensus.cc:3037] T 8da35f870a754219b104dce3b8d87846 P a07e4e0445a140de92c89b90c930e418 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:49.661123 27986 raft_consensus.cc:3037] T 8da35f870a754219b104dce3b8d87846 P 7895952636a34ed7b8d53efba58d8dbb [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:49.662248 27986 raft_consensus.cc:2454] T 8da35f870a754219b104dce3b8d87846 P 7895952636a34ed7b8d53efba58d8dbb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87492194659641e2a9141f7233bcc623 in term 1.
I20260810 18:48:49.662248 28153 raft_consensus.cc:2454] T 8da35f870a754219b104dce3b8d87846 P a07e4e0445a140de92c89b90c930e418 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87492194659641e2a9141f7233bcc623 in term 1.
I20260810 18:48:49.662559 27744 leader_election.cc:302] T 8da35f870a754219b104dce3b8d87846 P 87492194659641e2a9141f7233bcc623 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7895952636a34ed7b8d53efba58d8dbb, 87492194659641e2a9141f7233bcc623; no voters:
I20260810 18:48:49.662740 28307 raft_consensus.cc:2781] T 8da35f870a754219b104dce3b8d87846 P 87492194659641e2a9141f7233bcc623 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:49.662861 28307 raft_consensus.cc:686] T 8da35f870a754219b104dce3b8d87846 P 87492194659641e2a9141f7233bcc623 [term 1 LEADER]: Becoming Leader. State: Replica: 87492194659641e2a9141f7233bcc623, State: Running, Role: LEADER
I20260810 18:48:49.663017 28307 consensus_queue.cc:237] T 8da35f870a754219b104dce3b8d87846 P 87492194659641e2a9141f7233bcc623 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } } peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } }
I20260810 18:48:49.664651 27641 catalog_manager.cc:5495] T 8da35f870a754219b104dce3b8d87846 P 87492194659641e2a9141f7233bcc623 reported cstate change: term changed from 0 to 1, leader changed from <none> to 87492194659641e2a9141f7233bcc623 (127.18.168.129). New cstate: current_term: 1 leader_uuid: "87492194659641e2a9141f7233bcc623" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87492194659641e2a9141f7233bcc623" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 } health_report { overall_health: UNKNOWN } } }
I20260810 18:48:49.692617 28248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:49.689952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51346 (local address 127.18.168.129:43315)
0810 18:48:49.690094 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:49.690097 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:49.690110 (+ 13us) server_negotiation.cc:407] Connection header received
0810 18:48:49.690197 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:49.690199 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:49.690238 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:49.690297 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:48:49.690455 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.690943 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.691701 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.691916 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.691998 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:49.692211 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:48:49.692279 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:48:49.692331 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:49.692410 (+ 79us) server_negotiation.cc:299] Negotiation successful
0810 18:48:49.692450 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 18:48:49.698411 28153 raft_consensus.cc:1261] T 8da35f870a754219b104dce3b8d87846 P a07e4e0445a140de92c89b90c930e418 [term 1 FOLLOWER]: Refusing update from remote peer 87492194659641e2a9141f7233bcc623: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:49.698765 27986 raft_consensus.cc:1261] T 8da35f870a754219b104dce3b8d87846 P 7895952636a34ed7b8d53efba58d8dbb [term 1 FOLLOWER]: Refusing update from remote peer 87492194659641e2a9141f7233bcc623: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:49.698814 28260 consensus_queue.cc:1035] T 8da35f870a754219b104dce3b8d87846 P 87492194659641e2a9141f7233bcc623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a07e4e0445a140de92c89b90c930e418" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:49.699157 28260 consensus_queue.cc:1035] T 8da35f870a754219b104dce3b8d87846 P 87492194659641e2a9141f7233bcc623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7895952636a34ed7b8d53efba58d8dbb" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:49.730640 27641 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52602:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:49.730818 27641 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52602:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:49.732826 27641 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dfbc86f800924b13a83b5a0e342be613: Sending DeleteTablet for 3 replicas of tablet 8da35f870a754219b104dce3b8d87846
I20260810 18:48:49.733407 27962 tablet_service.cc:1514] Processing DeleteTablet for tablet 8da35f870a754219b104dce3b8d87846 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:49 UTC) from {username='oryx', principal='oryx/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:39090
I20260810 18:48:49.733405 27799 tablet_service.cc:1514] Processing DeleteTablet for tablet 8da35f870a754219b104dce3b8d87846 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:49 UTC) from {username='oryx', principal='oryx/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:51332
I20260810 18:48:49.733433 28132 tablet_service.cc:1514] Processing DeleteTablet for tablet 8da35f870a754219b104dce3b8d87846 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:49 UTC) from {username='oryx', principal='oryx/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:39218
I20260810 18:48:49.733609 28280 tablet_replica.cc:330] T 8da35f870a754219b104dce3b8d87846 P a07e4e0445a140de92c89b90c930e418: stopping tablet replica
I20260810 18:48:49.733711 28279 tablet_replica.cc:330] T 8da35f870a754219b104dce3b8d87846 P 87492194659641e2a9141f7233bcc623: stopping tablet replica
I20260810 18:48:49.733734 28280 raft_consensus.cc:2229] T 8da35f870a754219b104dce3b8d87846 P a07e4e0445a140de92c89b90c930e418 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:49.733867 28279 raft_consensus.cc:2229] T 8da35f870a754219b104dce3b8d87846 P 87492194659641e2a9141f7233bcc623 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:48:49.733914 28280 raft_consensus.cc:2258] T 8da35f870a754219b104dce3b8d87846 P a07e4e0445a140de92c89b90c930e418 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:49.734098 28279 raft_consensus.cc:2258] T 8da35f870a754219b104dce3b8d87846 P 87492194659641e2a9141f7233bcc623 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:49.734510 28281 tablet_replica.cc:330] T 8da35f870a754219b104dce3b8d87846 P 7895952636a34ed7b8d53efba58d8dbb: stopping tablet replica
I20260810 18:48:49.734651 28281 raft_consensus.cc:2229] T 8da35f870a754219b104dce3b8d87846 P 7895952636a34ed7b8d53efba58d8dbb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:49.734661 28280 ts_tablet_manager.cc:1905] T 8da35f870a754219b104dce3b8d87846 P a07e4e0445a140de92c89b90c930e418: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:49.734782 28281 raft_consensus.cc:2258] T 8da35f870a754219b104dce3b8d87846 P 7895952636a34ed7b8d53efba58d8dbb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:49.734861 28279 ts_tablet_manager.cc:1905] T 8da35f870a754219b104dce3b8d87846 P 87492194659641e2a9141f7233bcc623: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:49.735061 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 27723
I20260810 18:48:49.737826 28280 ts_tablet_manager.cc:1918] T 8da35f870a754219b104dce3b8d87846 P a07e4e0445a140de92c89b90c930e418: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:48:49.737926 28280 log.cc:1196] T 8da35f870a754219b104dce3b8d87846 P a07e4e0445a140de92c89b90c930e418: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-2/wal/wals/8da35f870a754219b104dce3b8d87846
I20260810 18:48:49.738181 28280 ts_tablet_manager.cc:1939] T 8da35f870a754219b104dce3b8d87846 P a07e4e0445a140de92c89b90c930e418: Deleting consensus metadata
I20260810 18:48:49.738828 28281 ts_tablet_manager.cc:1905] T 8da35f870a754219b104dce3b8d87846 P 7895952636a34ed7b8d53efba58d8dbb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:49.738902 27625 catalog_manager.cc:4841] TS a07e4e0445a140de92c89b90c930e418 (127.18.168.131:43327): tablet 8da35f870a754219b104dce3b8d87846 (table test-table [id=1b7fc65a81924aecbd0ad2b7a4ae0642]) successfully deleted
I20260810 18:48:49.740950 28281 ts_tablet_manager.cc:1918] T 8da35f870a754219b104dce3b8d87846 P 7895952636a34ed7b8d53efba58d8dbb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:48:49.741007 28281 log.cc:1196] T 8da35f870a754219b104dce3b8d87846 P 7895952636a34ed7b8d53efba58d8dbb: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387687857264-19106-0/minicluster-data/ts-1/wal/wals/8da35f870a754219b104dce3b8d87846
I20260810 18:48:49.741191 28281 ts_tablet_manager.cc:1939] T 8da35f870a754219b104dce3b8d87846 P 7895952636a34ed7b8d53efba58d8dbb: Deleting consensus metadata
I20260810 18:48:49.741882 27627 catalog_manager.cc:4841] TS 7895952636a34ed7b8d53efba58d8dbb (127.18.168.130:37203): tablet 8da35f870a754219b104dce3b8d87846 (table test-table [id=1b7fc65a81924aecbd0ad2b7a4ae0642]) successfully deleted
W20260810 18:48:49.746838 27625 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.168.129:43315 (error 108)
I20260810 18:48:49.747066 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 27885
I20260810 18:48:49.749962 28351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:49.749422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:43315 (local address 127.0.0.1:51354)
0810 18:48:49.749810 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:49.749883 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.129:43315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":258,"threads_started":1}
W20260810 18:48:49.750099 27625 catalog_manager.cc:4584] TS 87492194659641e2a9141f7233bcc623 (127.18.168.129:43315): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8da35f870a754219b104dce3b8d87846: Network error: Client connection negotiation failed: client connection to 127.18.168.129:43315: connect: Connection refused (error 111)
I20260810 18:48:49.756213 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 28053
I20260810 18:48:49.764652 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 27607
2026-08-10T18:48:49Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3223 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:48:49 dist-test-slave-cbsf krb5kdc[28354](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:48:49 dist-test-slave-cbsf krb5kdc[28354](info): set up 2 sockets
Aug 10 18:48:49 dist-test-slave-cbsf krb5kdc[28354](info): commencing operation
krb5kdc: starting...
W20260810 18:48:50.931936 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:48:51 dist-test-slave-cbsf krb5kdc[28354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:48:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:48:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.168.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
I20260810 18:48:51.063764 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:46373
--webserver_interface=127.18.168.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:38175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:46373
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:51.173928 28367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:51.174198 28367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:51.174307 28367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:51.178279 28367 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:48:51.178354 28367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:51.178392 28367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:51.178444 28367 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:48:51.178483 28367 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:48:51.183660 28367 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:38175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:46373
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:46373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:51.184736 28367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:51.185752 28367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:51.194135 28372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:51.194131 28373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:51.194269 28367 server_base.cc:1034] running on GCE node
W20260810 18:48:51.194132 28375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:51.194967 28367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:51.196030 28367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:51.197219 28367 hybrid_clock.cc:648] HybridClock initialized: now 1786387731197199 us; error 30 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-10T18:48:51Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1433 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:48:51 dist-test-slave-cbsf krb5kdc[28380](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:48:51 dist-test-slave-cbsf krb5kdc[28380](info): set up 2 sockets
Aug 10 18:48:51 dist-test-slave-cbsf krb5kdc[28380](info): commencing operation
krb5kdc: starting...
W20260810 18:48:52.315311 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:48:52 dist-test-slave-cbsf krb5kdc[28380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:48:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:48:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.168.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
I20260810 18:48:52.437841 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:41605
--webserver_interface=127.18.168.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:41875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:41605
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:52.550413 28393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:52.550668 28393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:52.550705 28393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:52.554687 28393 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:48:52.554751 28393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:52.554770 28393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:52.554785 28393 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:48:52.554800 28393 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:48:52.559805 28393 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:41875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:41605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:41605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:52.560667 28393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:52.561625 28393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:52.569638 28401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:52.569708 28404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:52.569694 28402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:52.569837 28393 server_base.cc:1034] running on GCE node
I20260810 18:48:52.570312 28393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:52.571395 28393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:52.572602 28393 hybrid_clock.cc:648] HybridClock initialized: now 1786387732572568 us; error 43 us; skew 500 ppm
Aug 10 18:48:52 dist-test-slave-cbsf krb5kdc[28380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387732, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:52.580808 28393 init.cc:377] Logged in from keytab as kudu/127.18.168.190@KRBTEST.COM (short username kudu)
I20260810 18:48:52.581679 28393 webserver.cc:460] Webserver started at http://127.18.168.190:45387/ using document root <none> and password file <none>
I20260810 18:48:52.582244 28393 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:52.582341 28393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:52.582587 28393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:52.584729 28393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "00d7560138004756aba2e321d7821d3d"
format_stamp: "Formatted at 2026-08-10 18:48:52 on dist-test-slave-cbsf"
server_key: "1fb76eac4630bb39b797702f72998e8a"
server_key_iv: "eb4a491d0165dff0a258ab97c0b44576"
server_key_version: "encryptionkey@0"
I20260810 18:48:52.585318 28393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "00d7560138004756aba2e321d7821d3d"
format_stamp: "Formatted at 2026-08-10 18:48:52 on dist-test-slave-cbsf"
server_key: "1fb76eac4630bb39b797702f72998e8a"
server_key_iv: "eb4a491d0165dff0a258ab97c0b44576"
server_key_version: "encryptionkey@0"
I20260810 18:48:52.589005 28393 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:48:52.591459 28413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:52.592814 28393 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:48:52.592950 28393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "00d7560138004756aba2e321d7821d3d"
format_stamp: "Formatted at 2026-08-10 18:48:52 on dist-test-slave-cbsf"
server_key: "1fb76eac4630bb39b797702f72998e8a"
server_key_iv: "eb4a491d0165dff0a258ab97c0b44576"
server_key_version: "encryptionkey@0"
I20260810 18:48:52.593056 28393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:52.623726 28393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:52.627069 28393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:52.627290 28393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:52.634356 28393 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:41605
I20260810 18:48:52.634383 28471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:41605 every 8 connection(s)
I20260810 18:48:52.635275 28393 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:48:52.638018 28472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:52.643414 28472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d: Bootstrap starting.
I20260810 18:48:52.644485 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 28393
I20260810 18:48:52.644636 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:48:52.644979 19106 external_mini_cluster.cc:1428] Setting key 359d44866c1a91139dbd5a0558b3a4a0
I20260810 18:48:52.646363 28472 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:52.647107 28472 log.cc:824] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:52.649170 28472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d: No bootstrap required, opened a new log
I20260810 18:48:52.652513 28472 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00d7560138004756aba2e321d7821d3d" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41605 } }
I20260810 18:48:52.652773 28472 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:52.652858 28472 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00d7560138004756aba2e321d7821d3d, State: Initialized, Role: FOLLOWER
I20260810 18:48:52.653324 28472 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00d7560138004756aba2e321d7821d3d" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41605 } }
I20260810 18:48:52.653510 28472 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:48:52.653584 28472 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:48:52.653678 28472 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d [term 0 FOLLOWER]: Advancing to term 1
Aug 10 18:48:52 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:52.654879 28472 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00d7560138004756aba2e321d7821d3d" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41605 } }
I20260810 18:48:52.655251 28472 leader_election.cc:302] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00d7560138004756aba2e321d7821d3d; no voters:
I20260810 18:48:52.655613 28472 leader_election.cc:288] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:48:52.655730 28479 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:52.655957 28479 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d [term 1 LEADER]: Becoming Leader. State: Replica: 00d7560138004756aba2e321d7821d3d, State: Running, Role: LEADER
I20260810 18:48:52.656323 28479 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00d7560138004756aba2e321d7821d3d" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41605 } }
I20260810 18:48:52.656886 28472 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:48:52.657655 28481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00d7560138004756aba2e321d7821d3d. Latest consensus state: current_term: 1 leader_uuid: "00d7560138004756aba2e321d7821d3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00d7560138004756aba2e321d7821d3d" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41605 } } }
I20260810 18:48:52.657681 28480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00d7560138004756aba2e321d7821d3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00d7560138004756aba2e321d7821d3d" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41605 } } }
I20260810 18:48:52.657830 28481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:52.657851 28480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:52.658227 28490 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:48:52.660634 28476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:52.646125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33464 (local address 127.18.168.190:41605)
0810 18:48:52.646644 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:52.646659 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:52.646694 (+ 35us) server_negotiation.cc:407] Connection header received
0810 18:48:52.647395 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:52.647415 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:52.647860 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:52.648226 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:52.649184 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:52.650229 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:52.651122 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:52.651595 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:52.651766 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:52.654718 (+ 2952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:52.654723 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:52.654735 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:52.654761 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:52.656967 (+ 2206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:52.657494 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:52.657501 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:52.657508 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:52.657669 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:52.658056 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:52.658059 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:52.658061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:52.658584 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:52.658800 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:52.659289 (+ 489us) server_negotiation.cc:299] Negotiation successful
0810 18:48:52.659651 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":142,"threads_started":1}
I20260810 18:48:52.662197 28490 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:48:52.667686 28490 catalog_manager.cc:1347] Generated new cluster ID: 5a0445ecf1da4dcab52967a4f1014958
I20260810 18:48:52.667759 28490 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:48:52.687593 28490 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:48:52.688553 28490 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:48:52.697111 28490 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d: Generated new TSK 0
I20260810 18:48:52.697974 28490 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.168.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab.
Entry for principal kudu/127.18.168.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab.
I20260810 18:48:52.728287 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.129:0
--local_ip_for_outbound_sockets=127.18.168.129
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:41605
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:41875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:52.845641 28503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:52.845886 28503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:52.845950 28503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:52.849825 28503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:52.849897 28503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:52.849975 28503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.129
I20260810 18:48:52.855185 28503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:41875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:41605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.129
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:52.856165 28503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:52.857272 28503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:52.866823 28512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:52.866794 28508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:52.866860 28509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:52.867058 28503 server_base.cc:1034] running on GCE node
I20260810 18:48:52.867460 28503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:52.868137 28503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:52.869304 28503 hybrid_clock.cc:648] HybridClock initialized: now 1786387732869290 us; error 26 us; skew 500 ppm
Aug 10 18:48:52 dist-test-slave-cbsf krb5kdc[28380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387732, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:52.872105 28503 init.cc:377] Logged in from keytab as kudu/127.18.168.129@KRBTEST.COM (short username kudu)
I20260810 18:48:52.873008 28503 webserver.cc:460] Webserver started at http://127.18.168.129:45597/ using document root <none> and password file <none>
I20260810 18:48:52.873574 28503 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:52.873670 28503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:52.873904 28503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:52.875888 28503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-0/data/instance:
uuid: "2198877dbbfe4c28945b76c2250f32d6"
format_stamp: "Formatted at 2026-08-10 18:48:52 on dist-test-slave-cbsf"
server_key: "f5734533bef4f9f5602f5fcba27dfd7c"
server_key_iv: "e9b00652460f8baf7521923dc925962a"
server_key_version: "encryptionkey@0"
I20260810 18:48:52.876458 28503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance:
uuid: "2198877dbbfe4c28945b76c2250f32d6"
format_stamp: "Formatted at 2026-08-10 18:48:52 on dist-test-slave-cbsf"
server_key: "f5734533bef4f9f5602f5fcba27dfd7c"
server_key_iv: "e9b00652460f8baf7521923dc925962a"
server_key_version: "encryptionkey@0"
I20260810 18:48:52.880107 28503 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:48:52.882539 28519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:52.883697 28503 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:48:52.883832 28503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-0/wal
uuid: "2198877dbbfe4c28945b76c2250f32d6"
format_stamp: "Formatted at 2026-08-10 18:48:52 on dist-test-slave-cbsf"
server_key: "f5734533bef4f9f5602f5fcba27dfd7c"
server_key_iv: "e9b00652460f8baf7521923dc925962a"
server_key_version: "encryptionkey@0"
I20260810 18:48:52.883941 28503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:52.906387 28503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:52.909190 28503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:52.909405 28503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:52.909978 28503 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:52.910796 28503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:52.910866 28503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:52.910935 28503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:52.910986 28503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:52.919878 28503 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.129:39417
I20260810 18:48:52.919901 28652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.129:39417 every 8 connection(s)
I20260810 18:48:52.920811 28503 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
I20260810 18:48:52.925228 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 28503
I20260810 18:48:52.925361 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance
I20260810 18:48:52.925663 19106 external_mini_cluster.cc:1428] Setting key df596f1994ded3df4a0575e18857d756
Aug 10 18:48:52 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387732, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:52.934957 28476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:52.922643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:47527 (local address 127.18.168.190:41605)
0810 18:48:52.922826 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:52.922829 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:52.923728 (+ 899us) server_negotiation.cc:407] Connection header received
0810 18:48:52.924580 (+ 852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:52.924583 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:52.924630 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:52.924753 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:52.926012 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:52.926570 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:52.927935 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:52.928381 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:52.928505 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:52.931366 (+ 2861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:52.931370 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:52.931372 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:52.931394 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:52.932913 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:52.933434 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:52.933443 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:52.933444 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:52.933519 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:52.933862 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:52.933867 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:52.933869 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:52.934049 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:52.934161 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:52.934658 (+ 497us) server_negotiation.cc:299] Negotiation successful
0810 18:48:52.934785 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 18:48:52.935776 28655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:52.922941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.129:47527)
0810 18:48:52.923580 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:52.923620 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:52.924400 (+ 780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:52.924898 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:52.924905 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:52.925237 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:52.925854 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:52.925869 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:52.926717 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:52.926721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:52.927796 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:52.927806 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:52.928543 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:52.928546 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:52.928683 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:52.929429 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:52.929456 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:52.931191 (+ 1735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:52.933033 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:52.933039 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:52.933052 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:52.933319 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:52.933612 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:52.933616 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:52.933618 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:52.933761 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:52.934169 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:52.934177 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:52.934412 (+ 235us) client_negotiation.cc:769] Sending connection context
0810 18:48:52.934606 (+ 194us) client_negotiation.cc:240] Negotiation successful
0810 18:48:52.934915 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":104,"threads_started":1}
WARNING: no policy specified for kudu/127.18.168.130@KRBTEST.COM; defaulting to no policy
I20260810 18:48:52.937038 28653 heartbeater.cc:347] Connected to a master server at 127.18.168.190:41605
I20260810 18:48:52.937307 28653 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:52.937762 28653 heartbeater.cc:511] Master 127.18.168.190:41605 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.130@KRBTEST.COM" created.
I20260810 18:48:52.939431 28432 ts_manager.cc:194] Registered new tserver with Master: 2198877dbbfe4c28945b76c2250f32d6 (127.18.168.129:39417)
I20260810 18:48:52.940541 28432 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.129@KRBTEST.COM'} at 127.18.168.129:47527
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab.
Entry for principal kudu/127.18.168.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab.
I20260810 18:48:52.957008 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.130:0
--local_ip_for_outbound_sockets=127.18.168.130
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:41605
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:41875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:53.074926 28659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:53.075166 28659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:53.075205 28659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:53.079037 28659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:53.079085 28659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:53.079137 28659 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.130
I20260810 18:48:53.084192 28659 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:41875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:41605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.130
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:53.085388 28659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:53.086341 28659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:53.095819 28668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:53.095850 28670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:53.095819 28665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:53.096127 28659 server_base.cc:1034] running on GCE node
I20260810 18:48:53.096477 28659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:53.097223 28659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:53.098404 28659 hybrid_clock.cc:648] HybridClock initialized: now 1786387733098372 us; error 43 us; skew 500 ppm
Aug 10 18:48:53 dist-test-slave-cbsf krb5kdc[28380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387733, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:53.101099 28659 init.cc:377] Logged in from keytab as kudu/127.18.168.130@KRBTEST.COM (short username kudu)
I20260810 18:48:53.101922 28659 webserver.cc:460] Webserver started at http://127.18.168.130:42823/ using document root <none> and password file <none>
I20260810 18:48:53.102496 28659 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:53.102588 28659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:53.102824 28659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:53.104828 28659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-1/data/instance:
uuid: "60386210f4774ceabb2965635b619ec4"
format_stamp: "Formatted at 2026-08-10 18:48:53 on dist-test-slave-cbsf"
server_key: "2f9f3190316e081f7954abbb87f51706"
server_key_iv: "6372d2a19b89f7be6fb8b9d7bd47ca52"
server_key_version: "encryptionkey@0"
I20260810 18:48:53.105410 28659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance:
uuid: "60386210f4774ceabb2965635b619ec4"
format_stamp: "Formatted at 2026-08-10 18:48:53 on dist-test-slave-cbsf"
server_key: "2f9f3190316e081f7954abbb87f51706"
server_key_iv: "6372d2a19b89f7be6fb8b9d7bd47ca52"
server_key_version: "encryptionkey@0"
I20260810 18:48:53.109103 28659 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 18:48:53.111510 28678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:53.112849 28659 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:48:53.113020 28659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-1/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-1/wal
uuid: "60386210f4774ceabb2965635b619ec4"
format_stamp: "Formatted at 2026-08-10 18:48:53 on dist-test-slave-cbsf"
server_key: "2f9f3190316e081f7954abbb87f51706"
server_key_iv: "6372d2a19b89f7be6fb8b9d7bd47ca52"
server_key_version: "encryptionkey@0"
I20260810 18:48:53.113143 28659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:53.130223 28659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:53.133289 28659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:53.133533 28659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:53.134195 28659 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:53.135157 28659 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:53.135241 28659 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:53.135321 28659 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:53.135385 28659 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:53.145800 28659 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.130:46213
I20260810 18:48:53.145821 28812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.130:46213 every 8 connection(s)
I20260810 18:48:53.146664 28659 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
I20260810 18:48:53.153767 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 28659
I20260810 18:48:53.153944 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance
I20260810 18:48:53.154241 19106 external_mini_cluster.cc:1428] Setting key 05b51bba1b442235537e8191addf3d2c
Aug 10 18:48:53 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387733, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:53.160753 28476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:53.148449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:56699 (local address 127.18.168.190:41605)
0810 18:48:53.148581 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:53.148585 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:53.149615 (+ 1030us) server_negotiation.cc:407] Connection header received
0810 18:48:53.150488 (+ 873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:53.150491 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:53.150551 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:53.150664 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:53.151773 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:53.152264 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:53.153480 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:53.153724 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:53.153831 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:53.156527 (+ 2696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:53.156531 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:53.156534 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:53.156566 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:53.158628 (+ 2062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:53.159212 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:53.159218 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:53.159223 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:53.159331 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:53.159670 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:53.159674 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:53.159676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:53.159870 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:53.159984 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:53.160447 (+ 463us) server_negotiation.cc:299] Negotiation successful
0810 18:48:53.160572 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 18:48:53.161589 28815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:53.148837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.130:56699)
0810 18:48:53.149467 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:53.149507 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:53.150286 (+ 779us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:53.150832 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:53.150839 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:53.151191 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:53.151636 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:53.151649 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:53.152381 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:53.152384 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:53.153342 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:53.153353 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:53.153883 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:53.153887 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:53.154098 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:53.154678 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:53.154693 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:53.156366 (+ 1673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:53.158771 (+ 2405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:53.158775 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:53.158786 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:53.159093 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:53.159434 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:53.159437 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:53.159438 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:53.159563 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:53.159984 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:53.159990 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:53.160218 (+ 228us) client_negotiation.cc:769] Sending connection context
0810 18:48:53.160402 (+ 184us) client_negotiation.cc:240] Negotiation successful
0810 18:48:53.160770 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":102,"threads_started":1}
I20260810 18:48:53.162833 28813 heartbeater.cc:347] Connected to a master server at 127.18.168.190:41605
I20260810 18:48:53.163097 28813 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:53.163693 28813 heartbeater.cc:511] Master 127.18.168.190:41605 requested a full tablet report, sending...
I20260810 18:48:53.164880 28432 ts_manager.cc:194] Registered new tserver with Master: 60386210f4774ceabb2965635b619ec4 (127.18.168.130:46213)
WARNING: no policy specified for kudu/127.18.168.131@KRBTEST.COM; defaulting to no policy
I20260810 18:48:53.165516 28432 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.130@KRBTEST.COM'} at 127.18.168.130:56699
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab.
Entry for principal kudu/127.18.168.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab.
I20260810 18:48:53.187152 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.131:0
--local_ip_for_outbound_sockets=127.18.168.131
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:41605
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:41875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:53.299901 28818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:53.300153 28818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:53.300196 28818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:53.304342 28818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:53.304395 28818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:53.304476 28818 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.131
I20260810 18:48:53.310252 28818 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:41875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:41605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.131
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:53.311276 28818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:53.312286 28818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:53.321923 28828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:53.321930 28825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:53.322026 28824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:53.322422 28818 server_base.cc:1034] running on GCE node
I20260810 18:48:53.322810 28818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:53.323410 28818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:53.324572 28818 hybrid_clock.cc:648] HybridClock initialized: now 1786387733324554 us; error 36 us; skew 500 ppm
Aug 10 18:48:53 dist-test-slave-cbsf krb5kdc[28380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387733, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:53.327327 28818 init.cc:377] Logged in from keytab as kudu/127.18.168.131@KRBTEST.COM (short username kudu)
I20260810 18:48:53.328243 28818 webserver.cc:460] Webserver started at http://127.18.168.131:44947/ using document root <none> and password file <none>
I20260810 18:48:53.328848 28818 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:53.328941 28818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:53.329193 28818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:53.331122 28818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-2/data/instance:
uuid: "2e3a8e215d2946a1b5baa909e152dd28"
format_stamp: "Formatted at 2026-08-10 18:48:53 on dist-test-slave-cbsf"
server_key: "b3a86789aac77f6999c90f7c585ddeb4"
server_key_iv: "579b9df14af18cab6cfc57e5386154a8"
server_key_version: "encryptionkey@0"
I20260810 18:48:53.331686 28818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance:
uuid: "2e3a8e215d2946a1b5baa909e152dd28"
format_stamp: "Formatted at 2026-08-10 18:48:53 on dist-test-slave-cbsf"
server_key: "b3a86789aac77f6999c90f7c585ddeb4"
server_key_iv: "579b9df14af18cab6cfc57e5386154a8"
server_key_version: "encryptionkey@0"
I20260810 18:48:53.335343 28818 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260810 18:48:53.337862 28836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:53.339071 28818 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260810 18:48:53.339216 28818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-2/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-2/wal
uuid: "2e3a8e215d2946a1b5baa909e152dd28"
format_stamp: "Formatted at 2026-08-10 18:48:53 on dist-test-slave-cbsf"
server_key: "b3a86789aac77f6999c90f7c585ddeb4"
server_key_iv: "579b9df14af18cab6cfc57e5386154a8"
server_key_version: "encryptionkey@0"
I20260810 18:48:53.339318 28818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:53.362479 28818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:53.365610 28818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:53.365834 28818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:53.366427 28818 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:53.367239 28818 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:53.367319 28818 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:53.367394 28818 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:53.367447 28818 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:53.377125 28818 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.131:43243
I20260810 18:48:53.377147 28970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.131:43243 every 8 connection(s)
I20260810 18:48:53.378006 28818 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
I20260810 18:48:53.384323 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 28818
I20260810 18:48:53.384474 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance
I20260810 18:48:53.384819 19106 external_mini_cluster.cc:1428] Setting key 99824da380ed5543b3e325567277f49e
Aug 10 18:48:53 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387733, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:48:53.392006 28476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:53.379978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:45427 (local address 127.18.168.190:41605)
0810 18:48:53.380115 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:53.380119 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:53.381122 (+ 1003us) server_negotiation.cc:407] Connection header received
0810 18:48:53.382015 (+ 893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:53.382021 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:53.382083 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:53.382200 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:53.383357 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:53.383943 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:53.385564 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:53.385895 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:53.386032 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:53.388619 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:53.388622 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:53.388624 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:53.388647 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:53.390003 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:53.390614 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:53.390616 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:53.390618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:53.390691 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:53.390990 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:53.390993 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:53.390994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:53.391186 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:53.391263 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:53.391796 (+ 533us) server_negotiation.cc:299] Negotiation successful
0810 18:48:53.391887 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 18:48:53.392951 28973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:53.380309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.131:45427)
0810 18:48:53.380979 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:53.381017 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:53.381802 (+ 785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:53.382336 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:53.382342 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:53.382766 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:53.383227 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:53.383240 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:53.384083 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:53.384088 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:53.385435 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:53.385456 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:53.386082 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:53.386087 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:53.386273 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:53.386923 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:53.386956 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:53.388444 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:53.390140 (+ 1696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:53.390145 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:53.390169 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:53.390520 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:53.390770 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:53.390772 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:53.390773 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:53.390907 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:53.391276 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:53.391281 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:53.391579 (+ 298us) client_negotiation.cc:769] Sending connection context
0810 18:48:53.391749 (+ 170us) client_negotiation.cc:240] Negotiation successful
0810 18:48:53.392001 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":116,"threads_started":1}
I20260810 18:48:53.394142 28971 heartbeater.cc:347] Connected to a master server at 127.18.168.190:41605
I20260810 18:48:53.394351 28971 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:53.394835 28971 heartbeater.cc:511] Master 127.18.168.190:41605 requested a full tablet report, sending...
I20260810 18:48:53.395855 28432 ts_manager.cc:194] Registered new tserver with Master: 2e3a8e215d2946a1b5baa909e152dd28 (127.18.168.131:43243)
I20260810 18:48:53.396445 28432 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.131@KRBTEST.COM'} at 127.18.168.131:45427
I20260810 18:48:53.398775 19106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.18.168.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.168.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.168.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.129.keytab.
Entry for principal oryx/127.18.168.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.129.keytab.
WARNING: no policy specified for oryx/127.18.168.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.168.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.168.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.130.keytab.
Entry for principal oryx/127.18.168.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.130.keytab.
WARNING: no policy specified for oryx/127.18.168.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.168.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.168.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.131.keytab.
Entry for principal oryx/127.18.168.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.131.keytab.
I20260810 18:48:53.476272 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 28503
I20260810 18:48:53.485282 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 28659
I20260810 18:48:53.495527 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 28818
I20260810 18:48:53.505247 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 28393
I20260810 18:48:53.513862 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:41605
--webserver_interface=127.18.168.190
--webserver_port=45387
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:41875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:41605
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:53.628995 28982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:53.629329 28982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:53.629421 28982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:53.633351 28982 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:48:53.633404 28982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:53.633431 28982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:53.633451 28982 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:48:53.633469 28982 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:48:53.638521 28982 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:41875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:41605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:41605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=45387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:53.639420 28982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:53.640654 28982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:53.649431 28990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:53.649360 28988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:53.649360 28987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:53.649549 28982 server_base.cc:1034] running on GCE node
I20260810 18:48:53.650206 28982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:53.651247 28982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:53.652412 28982 hybrid_clock.cc:648] HybridClock initialized: now 1786387733652395 us; error 30 us; skew 500 ppm
Aug 10 18:48:53 dist-test-slave-cbsf krb5kdc[28380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387733, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:53.655304 28982 init.cc:377] Logged in from keytab as kudu/127.18.168.190@KRBTEST.COM (short username kudu)
I20260810 18:48:53.656337 28982 webserver.cc:460] Webserver started at http://127.18.168.190:45387/ using document root <none> and password file <none>
I20260810 18:48:53.657047 28982 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:53.657146 28982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:53.660981 28982 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260810 18:48:53.663019 28999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:53.664227 28982 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:48:53.664373 28982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "00d7560138004756aba2e321d7821d3d"
format_stamp: "Formatted at 2026-08-10 18:48:52 on dist-test-slave-cbsf"
server_key: "1fb76eac4630bb39b797702f72998e8a"
server_key_iv: "eb4a491d0165dff0a258ab97c0b44576"
server_key_version: "encryptionkey@0"
I20260810 18:48:53.664980 28982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:53.705750 28982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:53.709146 28982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:53.709365 28982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:53.716780 28982 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:41605
I20260810 18:48:53.716812 29059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:41605 every 8 connection(s)
I20260810 18:48:53.717803 28982 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:48:53.720836 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 28982
I20260810 18:48:53.721283 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.129:39417
--local_ip_for_outbound_sockets=127.18.168.129
--tserver_master_addrs=127.18.168.190:41605
--webserver_port=45597
--webserver_interface=127.18.168.129
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:41875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.168.129
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.129.keytab with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:48:53.722182 29060 sys_catalog.cc:263] Verifying existing consensus state
I20260810 18:48:53.723999 29060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d: Bootstrap starting.
I20260810 18:48:53.748023 29060 log.cc:824] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:53.756533 29060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:48:53.757028 29060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d: Bootstrap complete.
I20260810 18:48:53.762838 29060 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00d7560138004756aba2e321d7821d3d" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41605 } }
I20260810 18:48:53.763377 29060 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00d7560138004756aba2e321d7821d3d, State: Initialized, Role: FOLLOWER
I20260810 18:48:53.763909 29060 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00d7560138004756aba2e321d7821d3d" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41605 } }
I20260810 18:48:53.764053 29060 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:48:53.764122 29060 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:48:53.764236 29060 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d [term 1 FOLLOWER]: Advancing to term 2
I20260810 18:48:53.766870 29060 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00d7560138004756aba2e321d7821d3d" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41605 } }
I20260810 18:48:53.767251 29060 leader_election.cc:302] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00d7560138004756aba2e321d7821d3d; no voters:
I20260810 18:48:53.767611 29060 leader_election.cc:288] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 18:48:53.767778 29064 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d [term 2 FOLLOWER]: Leader election won for term 2
I20260810 18:48:53.768217 29064 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d [term 2 LEADER]: Becoming Leader. State: Replica: 00d7560138004756aba2e321d7821d3d, State: Running, Role: LEADER
I20260810 18:48:53.768644 29064 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00d7560138004756aba2e321d7821d3d" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41605 } }
I20260810 18:48:53.768877 29060 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:48:53.770268 29066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00d7560138004756aba2e321d7821d3d. Latest consensus state: current_term: 2 leader_uuid: "00d7560138004756aba2e321d7821d3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00d7560138004756aba2e321d7821d3d" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41605 } } }
I20260810 18:48:53.770421 29066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:53.771193 29065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "00d7560138004756aba2e321d7821d3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00d7560138004756aba2e321d7821d3d" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41605 } } }
I20260810 18:48:53.771282 29065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:53.771345 29076 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:48:53.774580 29076 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:48:53.775238 29076 catalog_manager.cc:1259] Loaded cluster ID: 5a0445ecf1da4dcab52967a4f1014958
I20260810 18:48:53.775296 29076 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:48:53.777601 29076 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:48:53.778290 29076 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 00d7560138004756aba2e321d7821d3d: Loaded TSK: 0
I20260810 18:48:53.779471 29076 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 18:48:53.852758 29062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:53.853003 29062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:53.853044 29062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:53.857077 29062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:53.857128 29062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:53.857230 29062 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.129
I20260810 18:48:53.862438 29062 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:41875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.129.keytab
--principal=oryx/127.18.168.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.129:39417
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.168.129
--webserver_port=45597
--tserver_master_addrs=127.18.168.190:41605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.129
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:53.863409 29062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:53.864401 29062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:53.874557 29091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:53.874529 29090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:53.874600 29094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:53.875020 29062 server_base.cc:1034] running on GCE node
I20260810 18:48:53.875422 29062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:53.876113 29062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:53.877296 29062 hybrid_clock.cc:648] HybridClock initialized: now 1786387733877267 us; error 44 us; skew 500 ppm
Aug 10 18:48:53 dist-test-slave-cbsf krb5kdc[28380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387733, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.168.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:53.880196 29062 init.cc:377] Logged in from keytab as oryx/127.18.168.129@KRBTEST.COM (short username oryx)
I20260810 18:48:53.881248 29062 webserver.cc:460] Webserver started at http://127.18.168.129:45597/ using document root <none> and password file <none>
I20260810 18:48:53.881857 29062 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:53.881935 29062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:53.885847 29062 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:48:53.888006 29101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:53.889398 29062 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:48:53.889494 29062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-0/wal
uuid: "2198877dbbfe4c28945b76c2250f32d6"
format_stamp: "Formatted at 2026-08-10 18:48:52 on dist-test-slave-cbsf"
server_key: "f5734533bef4f9f5602f5fcba27dfd7c"
server_key_iv: "e9b00652460f8baf7521923dc925962a"
server_key_version: "encryptionkey@0"
I20260810 18:48:53.890061 29062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:53.912475 29062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:53.915716 29062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:53.915930 29062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:53.916566 29062 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:53.917461 29062 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:53.917538 29062 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:53.917613 29062 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:53.917665 29062 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:53.927716 29062 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.129:39417
I20260810 18:48:53.927742 29235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.129:39417 every 8 connection(s)
I20260810 18:48:53.928735 29062 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
I20260810 18:48:53.929674 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 29062
I20260810 18:48:53.930107 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.130:46213
--local_ip_for_outbound_sockets=127.18.168.130
--tserver_master_addrs=127.18.168.190:41605
--webserver_port=42823
--webserver_interface=127.18.168.130
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:41875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.168.130
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.130.keytab with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 18:48:53 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.129@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:48:53 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.129@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
W20260810 18:48:53.942353 29238 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:53.930816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:59379 (local address 127.18.168.190:41605)
0810 18:48:53.931366 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:53.931388 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:53.932803 (+ 1415us) server_negotiation.cc:407] Connection header received
0810 18:48:53.933066 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:53.933075 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:53.933343 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:53.933658 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:53.934778 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:53.935772 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:53.936984 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:53.937461 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:53.937638 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:53.941661 (+ 4023us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.129:59379: BlockingRecv error: recv got EOF from 127.18.168.129:59379 (error 108)
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":133,"threads_started":1}
I20260810 18:48:53.942432 29239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:53.930996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.129:59379)
0810 18:48:53.931743 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:53.931794 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:53.932609 (+ 815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:53.933731 (+ 1122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:53.933739 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:53.934161 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:53.934602 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:53.934615 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:53.935907 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:53.935910 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:53.936855 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:53.936865 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:53.937625 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:53.937629 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:53.937819 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:53.938408 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:53.938424 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:53.941461 (+ 3037us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":127,"threads_started":1}
W20260810 18:48:53.943034 29236 heartbeater.cc:644] Failed to heartbeat to 127.18.168.190:41605 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.168.190:41605: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Aug 10 18:48:53 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.129@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:48:53 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.129@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:48:53.948855 29239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:53.943455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.129:41121)
0810 18:48:53.943588 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:53.943602 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:53.943708 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:53.943939 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:53.943942 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:53.944103 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:53.944220 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:53.944226 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:53.944960 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:53.944963 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:53.945874 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:53.945885 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:53.946312 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:53.946315 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:53.946410 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:53.946856 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:53.946870 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:53.948675 (+ 1805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":61}
W20260810 18:48:53.949106 29238 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:53.943577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:41121 (local address 127.18.168.190:41605)
0810 18:48:53.943679 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:53.943682 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:53.943694 (+ 12us) server_negotiation.cc:407] Connection header received
0810 18:48:53.943794 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:53.943797 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:53.943834 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:53.943898 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:53.944328 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:53.944861 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:53.946000 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:53.946206 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:53.946283 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:53.948868 (+ 2585us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.129:41121: BlockingRecv error: recv got EOF from 127.18.168.129:41121 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 10 18:48:53 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.129@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:48:53 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.129@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:48:53.954283 29239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:53.949372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.129:45065)
0810 18:48:53.949636 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:53.949649 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:53.949761 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:53.950007 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:53.950010 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:53.950160 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:53.950224 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:53.950228 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:53.950938 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:53.950942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:53.951463 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:53.951470 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:53.951920 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:53.951924 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:53.952003 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:53.952463 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:53.952483 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:53.954155 (+ 1672us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196}
W20260810 18:48:53.954370 29238 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:53.949501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:45065 (local address 127.18.168.190:41605)
0810 18:48:53.949612 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:53.949616 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:53.949723 (+ 107us) server_negotiation.cc:407] Connection header received
0810 18:48:53.949861 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:53.949864 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:53.949904 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:53.949993 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:53.950340 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:53.950829 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:53.951582 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:53.951810 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:53.951909 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:53.954263 (+ 2354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.129:45065: BlockingRecv error: recv got EOF from 127.18.168.129:45065 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260810 18:48:53.954753 29236 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 18:48:54.046130 29240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:54.046445 29240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:54.046554 29240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:54.050501 29240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:54.050583 29240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:54.050676 29240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.130
I20260810 18:48:54.055712 29240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:41875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.130.keytab
--principal=oryx/127.18.168.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.130:46213
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.168.130
--webserver_port=42823
--tserver_master_addrs=127.18.168.190:41605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.130
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:54.056846 29240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:54.057849 29240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:54.067328 29249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:54.067333 29246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:54.067564 29240 server_base.cc:1034] running on GCE node
W20260810 18:48:54.067333 29247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:54.068212 29240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:54.068993 29240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:54.070149 29240 hybrid_clock.cc:648] HybridClock initialized: now 1786387734070135 us; error 25 us; skew 500 ppm
Aug 10 18:48:54 dist-test-slave-cbsf krb5kdc[28380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387734, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.168.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:54.073007 29240 init.cc:377] Logged in from keytab as oryx/127.18.168.130@KRBTEST.COM (short username oryx)
I20260810 18:48:54.073904 29240 webserver.cc:460] Webserver started at http://127.18.168.130:42823/ using document root <none> and password file <none>
I20260810 18:48:54.074473 29240 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:54.074569 29240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:54.078214 29240 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260810 18:48:54.080178 29256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:54.081367 29240 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260810 18:48:54.081588 29240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-1/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-1/wal
uuid: "60386210f4774ceabb2965635b619ec4"
format_stamp: "Formatted at 2026-08-10 18:48:53 on dist-test-slave-cbsf"
server_key: "2f9f3190316e081f7954abbb87f51706"
server_key_iv: "6372d2a19b89f7be6fb8b9d7bd47ca52"
server_key_version: "encryptionkey@0"
I20260810 18:48:54.082103 29240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:54.113332 29240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:54.116375 29240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:54.116616 29240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:54.117247 29240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:54.118074 29240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:54.118145 29240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:54.118216 29240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:54.118258 29240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:54.127471 29240 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.130:46213
I20260810 18:48:54.127490 29383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.130:46213 every 8 connection(s)
I20260810 18:48:54.128417 29240 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
Aug 10 18:48:54 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.130@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:48:54.137878 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 29240
I20260810 18:48:54.138321 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.131:43243
--local_ip_for_outbound_sockets=127.18.168.131
--tserver_master_addrs=127.18.168.190:41605
--webserver_port=44947
--webserver_interface=127.18.168.131
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:41875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.168.131
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.131.keytab with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 18:48:54 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.130@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
W20260810 18:48:54.139478 29238 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:54.130448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:45377 (local address 127.18.168.190:41605)
0810 18:48:54.130595 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:54.130599 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:54.131673 (+ 1074us) server_negotiation.cc:407] Connection header received
0810 18:48:54.132580 (+ 907us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:54.132584 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:54.132647 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:54.132773 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:54.133915 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:54.134452 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:54.135603 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:54.135838 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:54.135955 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:54.139359 (+ 3404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.130:45377: BlockingRecv error: recv got EOF from 127.18.168.130:45377 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 18:48:54.140089 29386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:54.130833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.130:45377)
0810 18:48:54.131525 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:54.131565 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:54.132342 (+ 777us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:54.132941 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:54.132948 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:54.133298 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:54.133760 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:54.133773 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:54.134585 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:54.134589 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:54.135456 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:54.135465 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:54.135992 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:54.135998 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:54.136174 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:54.136790 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:54.136806 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:54.139227 (+ 2421us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":108,"threads_started":1}
W20260810 18:48:54.140556 29384 heartbeater.cc:644] Failed to heartbeat to 127.18.168.190:41605 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.168.190:41605: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Aug 10 18:48:54 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.130@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:48:54 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.130@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:48:54.146098 29386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:54.141052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.130:59659)
0810 18:48:54.141263 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:54.141276 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:54.141376 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:54.141600 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:54.141602 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:54.141756 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:54.141837 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:54.141842 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:54.142531 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:54.142535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:54.143087 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:54.143094 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:54.143540 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:54.143543 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:54.143635 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:54.144105 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:54.144121 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:54.145943 (+ 1822us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142}
W20260810 18:48:54.146129 29238 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:54.141132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:59659 (local address 127.18.168.190:41605)
0810 18:48:54.141241 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:54.141245 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:54.141347 (+ 102us) server_negotiation.cc:407] Connection header received
0810 18:48:54.141452 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:54.141454 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:54.141494 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:54.141585 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:54.141948 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:54.142415 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:54.143206 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:54.143427 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:54.143539 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:54.146048 (+ 2509us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.130:59659: BlockingRecv error: recv got EOF from 127.18.168.130:59659 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 10 18:48:54 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.130@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:48:54 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.130@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:48:54.151849 29386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:54.146620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.130:41799)
0810 18:48:54.146778 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:54.146792 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:54.146879 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:54.147138 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:54.147143 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:54.147321 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:54.147428 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:54.147436 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:54.148064 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:54.148069 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:54.149069 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:54.149082 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:54.149477 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:54.149481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:54.149581 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:54.149973 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:54.149990 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:54.151719 (+ 1729us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":91}
W20260810 18:48:54.151928 29238 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:54.146677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:41799 (local address 127.18.168.190:41605)
0810 18:48:54.146771 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:54.146774 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:54.146835 (+ 61us) server_negotiation.cc:407] Connection header received
0810 18:48:54.146995 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:54.146997 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:54.147027 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:54.147087 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:54.147541 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:54.147944 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:54.149184 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:54.149366 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:54.149439 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:54.151819 (+ 2380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.130:41799: BlockingRecv error: recv got EOF from 127.18.168.130:41799 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260810 18:48:54.152247 29384 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 18:48:54.253249 29387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:54.253513 29387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:54.253582 29387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:54.257761 29387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:54.257834 29387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:54.257920 29387 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.131
I20260810 18:48:54.263068 29387 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:41875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/krb5kdc/oryx_127.18.168.131.keytab
--principal=oryx/127.18.168.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.131:43243
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.168.131
--webserver_port=44947
--tserver_master_addrs=127.18.168.190:41605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29387
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.131
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:48:54.264087 29387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:54.265157 29387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:54.275051 29394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:54.275035 29393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:54.275035 29396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:54.275643 29387 server_base.cc:1034] running on GCE node
I20260810 18:48:54.276005 29387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:54.276607 29387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:54.277808 29387 hybrid_clock.cc:648] HybridClock initialized: now 1786387734277793 us; error 26 us; skew 500 ppm
Aug 10 18:48:54 dist-test-slave-cbsf krb5kdc[28380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387734, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.168.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:54.280598 29387 init.cc:377] Logged in from keytab as oryx/127.18.168.131@KRBTEST.COM (short username oryx)
I20260810 18:48:54.281518 29387 webserver.cc:460] Webserver started at http://127.18.168.131:44947/ using document root <none> and password file <none>
I20260810 18:48:54.282086 29387 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:54.282187 29387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:54.286154 29387 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:48:54.288127 29404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:54.289372 29387 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:48:54.289521 29387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-2/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-2/wal
uuid: "2e3a8e215d2946a1b5baa909e152dd28"
format_stamp: "Formatted at 2026-08-10 18:48:53 on dist-test-slave-cbsf"
server_key: "b3a86789aac77f6999c90f7c585ddeb4"
server_key_iv: "579b9df14af18cab6cfc57e5386154a8"
server_key_version: "encryptionkey@0"
I20260810 18:48:54.290040 29387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:54.311389 29387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:54.314172 29387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:54.314374 29387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:54.314976 29387 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:54.315783 29387 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:54.315830 29387 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:54.315897 29387 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:54.315932 29387 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:54.324767 29387 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.131:43243
I20260810 18:48:54.324795 29542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.131:43243 every 8 connection(s)
I20260810 18:48:54.325727 29387 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
Aug 10 18:48:54 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.131@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:48:54 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.131@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:48:54.335767 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 29387
W20260810 18:48:54.335871 29238 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:54.327475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:45073 (local address 127.18.168.190:41605)
0810 18:48:54.327602 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:54.327608 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:54.328612 (+ 1004us) server_negotiation.cc:407] Connection header received
0810 18:48:54.329497 (+ 885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:54.329500 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:54.329548 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:54.329620 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:54.330736 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:54.331252 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:54.332356 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:54.332567 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:54.332642 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:54.335749 (+ 3107us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.131:45073: BlockingRecv error: recv got EOF from 127.18.168.131:45073 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 18:48:54.336438 29545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:54.327775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.131:45073)
0810 18:48:54.328428 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:54.328466 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:54.329318 (+ 852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:54.329785 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:54.329792 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:54.330122 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:54.330609 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:54.330623 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:54.331354 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:54.331356 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:54.332250 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:54.332259 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:54.332734 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:54.332741 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:54.332894 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:54.333370 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:54.333385 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:54.335644 (+ 2259us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":110,"threads_started":1}
W20260810 18:48:54.337195 29543 heartbeater.cc:644] Failed to heartbeat to 127.18.168.190:41605 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.168.190:41605: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Aug 10 18:48:54 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.131@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:48:54 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.131@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:48:54.342824 29545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:54.337699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.131:37419)
0810 18:48:54.337861 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:54.337875 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:54.337951 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:54.338166 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:54.338169 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:54.338317 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:54.338421 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:54.338428 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:54.339200 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:54.339206 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:54.339733 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:54.339741 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:54.340210 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:54.340213 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:54.340308 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:54.340892 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:54.340905 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:54.342685 (+ 1780us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100}
W20260810 18:48:54.342990 29547 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:54.337775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:37419 (local address 127.18.168.190:41605)
0810 18:48:54.337978 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:54.337981 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:54.337992 (+ 11us) server_negotiation.cc:407] Connection header received
0810 18:48:54.338022 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:54.338024 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:54.338053 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:54.338122 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:54.338555 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:54.339072 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:54.339858 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:54.340099 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:54.340208 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:54.342816 (+ 2608us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.131:37419: BlockingRecv error: recv got EOF from 127.18.168.131:37419 (error 108)
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":81,"threads_started":1}
I20260810 18:48:54.346506 29238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:54.336828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33478 (local address 127.18.168.190:41605)
0810 18:48:54.336954 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:54.336957 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:54.337195 (+ 238us) server_negotiation.cc:407] Connection header received
0810 18:48:54.337347 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:54.337350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:54.337393 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:54.337475 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:54.338455 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:54.338914 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:54.339792 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:54.340003 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:54.340111 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:54.341558 (+ 1447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:54.341563 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:54.341577 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:54.341603 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:54.344146 (+ 2543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:54.344552 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:54.344559 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:54.344565 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:54.344751 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:54.345074 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:54.345077 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:54.345079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:54.345564 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:54.345766 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:54.346068 (+ 302us) server_negotiation.cc:299] Negotiation successful
0810 18:48:54.346338 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Aug 10 18:48:54 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.131@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:48:54 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.131@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:48:54.348869 29545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:54.343432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.131:33619)
0810 18:48:54.343519 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:54.343533 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:54.343601 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:54.343852 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:54.343854 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:54.343972 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:54.344033 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:54.344037 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:54.344997 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:54.345002 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:54.345912 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:54.345922 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:54.346435 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:54.346447 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:54.346537 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:54.346942 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:54.346957 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:54.348743 (+ 1786us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":25}
W20260810 18:48:54.349025 29543 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 18:48:54.348954 29547 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:54.343517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:33619 (local address 127.18.168.190:41605)
0810 18:48:54.343649 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:54.343653 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:54.343667 (+ 14us) server_negotiation.cc:407] Connection header received
0810 18:48:54.343702 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:54.343705 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:54.343747 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:54.343842 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:54.344142 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:54.344888 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:54.346056 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:54.346320 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:54.346400 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:54.348853 (+ 2453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.131:33619: BlockingRecv error: recv got EOF from 127.18.168.131:33619 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 10 18:48:54 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.129@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:48:54 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.129@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:48:54.961516 29548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:54.955643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.129:48547)
0810 18:48:54.955937 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:54.955950 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:54.956043 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:54.956274 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:54.956276 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:54.956457 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:54.956577 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:54.956582 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:54.957395 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:54.957397 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:54.957961 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:54.957968 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:54.958386 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:54.958409 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:54.958558 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:54.959065 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:54.959077 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:54.961353 (+ 2276us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":95,"threads_started":1}
W20260810 18:48:54.961583 29549 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:54.955719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:48547 (local address 127.18.168.190:41605)
0810 18:48:54.956040 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:54.956043 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:54.956059 (+ 16us) server_negotiation.cc:407] Connection header received
0810 18:48:54.956103 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:54.956105 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:54.956149 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:54.956240 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:54.956712 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:54.957290 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:54.958067 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:54.958282 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:54.958367 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:54.961467 (+ 3100us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.129:48547: BlockingRecv error: recv got EOF from 127.18.168.129:48547 (error 108)
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":144,"threads_started":1}
Aug 10 18:48:55 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.130@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:48:55 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.130@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:48:55.158195 29550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:55.152935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.130:46241)
0810 18:48:55.153222 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:55.153237 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:55.153345 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:55.153566 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:55.153569 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:55.153749 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:55.153868 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:55.153873 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:55.154573 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:55.154575 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:55.155109 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:55.155116 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:55.155539 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:55.155542 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:55.155612 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:55.156079 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:55.156090 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:55.158065 (+ 1975us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":102,"threads_started":1}
W20260810 18:48:55.158361 29549 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:55.153004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:46241 (local address 127.18.168.190:41605)
0810 18:48:55.153160 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:55.153164 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:55.153317 (+ 153us) server_negotiation.cc:407] Connection header received
0810 18:48:55.153411 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:55.153414 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:55.153457 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:55.153544 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:55.153967 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:55.154481 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:55.155212 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:55.155445 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:55.155517 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:55.158197 (+ 2680us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.130:46241: BlockingRecv error: recv got EOF from 127.18.168.130:46241 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 10 18:48:55 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.131@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:48:55 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.131@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:48:55.354910 29551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:55.349642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.131:38661)
0810 18:48:55.349950 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:55.349963 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:55.350073 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:55.350297 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:55.350300 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:55.350474 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:55.350571 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:55.350576 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:55.351303 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:55.351305 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:55.351878 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:55.351886 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:55.352292 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:55.352295 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:55.352359 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:55.352867 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:55.352881 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:55.354771 (+ 1890us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":87,"threads_started":1}
W20260810 18:48:55.355108 29549 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:55.349821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:38661 (local address 127.18.168.190:41605)
0810 18:48:55.349943 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:55.349947 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:55.350015 (+ 68us) server_negotiation.cc:407] Connection header received
0810 18:48:55.350143 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:55.350145 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:55.350194 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:55.350260 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:55.350676 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:55.351208 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:55.351979 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:55.352198 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:55.352274 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:55.354930 (+ 2656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.131:38661: BlockingRecv error: recv got EOF from 127.18.168.131:38661 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 10 18:48:55 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.129@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:48:55 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.129@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:48:55.968056 29552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:55.962374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.129:47729)
0810 18:48:55.962692 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:55.962706 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:55.962804 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:55.963070 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:55.963072 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:55.963265 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:55.963366 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:55.963371 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:55.964264 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:55.964267 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:55.964923 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:55.964932 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:55.965324 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:55.965326 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:55.965386 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:55.965935 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:55.965947 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:55.967883 (+ 1936us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":101,"threads_started":1}
W20260810 18:48:55.968201 29553 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:55.962477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:47729 (local address 127.18.168.190:41605)
0810 18:48:55.962779 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:55.962782 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:55.962797 (+ 15us) server_negotiation.cc:407] Connection header received
0810 18:48:55.962898 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:55.962900 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:55.962951 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:55.963030 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:55.963476 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:55.964160 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:55.965021 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:55.965231 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:55.965303 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:55.968034 (+ 2731us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.129:47729: BlockingRecv error: recv got EOF from 127.18.168.129:47729 (error 108)
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":118,"threads_started":1}
Aug 10 18:48:56 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.130@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:48:56 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.130@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:48:56.164402 29554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:56.159109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.130:54089)
0810 18:48:56.159381 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:56.159395 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:56.159492 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:56.159745 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:56.159748 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:56.159923 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:56.160019 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:56.160024 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.160791 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.160794 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:56.161447 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:56.161457 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.161895 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.161897 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:56.161958 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:56.162472 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:56.162484 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:56.164290 (+ 1806us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":84,"threads_started":1}
W20260810 18:48:56.164600 29553 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:56.159167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:54089 (local address 127.18.168.190:41605)
0810 18:48:56.159310 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:56.159314 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:56.159446 (+ 132us) server_negotiation.cc:407] Connection header received
0810 18:48:56.159583 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:56.159585 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:56.159625 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:56.159704 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:56.160123 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.160633 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.161555 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.161797 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.161877 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:56.164427 (+ 2550us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.130:54089: BlockingRecv error: recv got EOF from 127.18.168.130:54089 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 10 18:48:56 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.131@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:48:56 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.131@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:48:56.361099 29555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:56.355656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.131:50727)
0810 18:48:56.356023 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:56.356037 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:56.356144 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:56.356424 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:56.356426 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:56.356625 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:56.356829 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:56.356838 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.357576 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.357579 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:56.358136 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:56.358144 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.358547 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.358550 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:56.358615 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:56.359106 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:56.359118 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:56.360947 (+ 1829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":110,"threads_started":1}
W20260810 18:48:56.361178 29553 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:56.355870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:50727 (local address 127.18.168.190:41605)
0810 18:48:56.356022 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:56.356026 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:56.356110 (+ 84us) server_negotiation.cc:407] Connection header received
0810 18:48:56.356229 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:56.356232 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:56.356293 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:56.356361 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:56.356937 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.357471 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.358236 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.358447 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.358523 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:56.361082 (+ 2559us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.131:50727: BlockingRecv error: recv got EOF from 127.18.168.131:50727 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 10 18:48:56 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.129@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:48:56 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.129@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:48:56.974987 29557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:56.969032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.129:36497)
0810 18:48:56.969396 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:56.969409 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:56.969541 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:56.969780 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:56.969782 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:56.969997 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:56.970166 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:56.970174 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.970984 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.970987 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:56.971624 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:56.971634 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.972085 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.972092 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:56.972187 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:56.972840 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:56.972852 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:56.974847 (+ 1995us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":174,"threads_started":1}
W20260810 18:48:56.975164 29556 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:56.969098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:36497 (local address 127.18.168.190:41605)
0810 18:48:56.969408 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:56.969412 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:56.969512 (+ 100us) server_negotiation.cc:407] Connection header received
0810 18:48:56.969612 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:56.969614 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:56.969663 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:56.969742 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:56.970280 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.970883 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.971745 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.971982 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.972062 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:56.974997 (+ 2935us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.129:36497: BlockingRecv error: recv got EOF from 127.18.168.129:36497 (error 108)
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":86,"threads_started":1}
Aug 10 18:48:57 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.130@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:48:57 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.130@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:48:57.171289 29558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:57.165155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.130:36695)
0810 18:48:57.165507 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:57.165530 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:57.165677 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:57.165964 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:57.165967 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:57.166199 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:57.166299 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:57.166305 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:57.167090 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:57.167096 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:57.167684 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:57.167693 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:57.168204 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:57.168207 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:57.168322 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:57.168953 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:57.168988 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:57.171164 (+ 2176us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":94,"threads_started":1}
W20260810 18:48:57.171474 29556 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:57.165290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:36695 (local address 127.18.168.190:41605)
0810 18:48:57.165465 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:57.165471 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:57.165608 (+ 137us) server_negotiation.cc:407] Connection header received
0810 18:48:57.165799 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:57.165802 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:57.165848 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:57.165939 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:57.166425 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:57.166970 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:57.167808 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:57.168076 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:57.168171 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:57.171311 (+ 3140us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.130:36695: BlockingRecv error: recv got EOF from 127.18.168.130:36695 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 10 18:48:57 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.131@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:48:57 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.131@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:48:57.367751 29559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:57.362222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.131:41819)
0810 18:48:57.362487 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:57.362500 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:57.362601 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:57.362816 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:57.362818 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:57.362998 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:57.363112 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:57.363117 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:57.363953 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:57.363956 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:57.364505 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:57.364512 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:57.364945 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:57.364948 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:57.365010 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:57.365504 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:57.365528 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:57.367585 (+ 2057us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":74,"threads_started":1}
W20260810 18:48:57.367797 29556 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:57.362351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:41819 (local address 127.18.168.190:41605)
0810 18:48:57.362517 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:57.362521 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:57.362553 (+ 32us) server_negotiation.cc:407] Connection header received
0810 18:48:57.362669 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:57.362671 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:57.362713 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:57.362776 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:57.363223 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:57.363825 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:57.364606 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:57.364851 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:57.364928 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:57.367699 (+ 2771us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.131:41819: BlockingRecv error: recv got EOF from 127.18.168.131:41819 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 10 18:48:57 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.129@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:48:57 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.129@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:48:57.986724 29561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:57.975892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.129:47469)
0810 18:48:57.976259 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:57.976284 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:57.976411 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:57.976751 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:57.976754 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:57.976994 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:57.977142 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:57.977168 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:57.977920 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:57.977922 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:57.978491 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:57.978498 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:57.978907 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:57.978910 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:57.978976 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:57.984332 (+ 5356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:57.984346 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:57.986602 (+ 2256us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":83,"threads_started":1}
W20260810 18:48:57.987051 29563 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:57.975927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:47469 (local address 127.18.168.190:41605)
0810 18:48:57.976361 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:57.976364 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:57.976376 (+ 12us) server_negotiation.cc:407] Connection header received
0810 18:48:57.976550 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:57.976553 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:57.976615 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:57.976745 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:57.977291 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:57.977824 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:57.978590 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:57.978802 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:57.978876 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:57.986760 (+ 7884us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.129:47469: BlockingRecv error: recv got EOF from 127.18.168.129:47469 (error 108)
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":59,"threads_started":1}
Aug 10 18:48:58 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.130@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:48:58 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.130@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:48:58.177768 29565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:58.172078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.130:33519)
0810 18:48:58.172357 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:58.172374 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:58.172488 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:58.172745 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:58.172748 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:58.172987 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:58.173092 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:58.173099 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:58.174186 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:58.174189 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:58.174781 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:58.174791 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:58.175211 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:58.175213 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:58.175275 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:58.175784 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:58.175796 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:58.177644 (+ 1848us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":114,"threads_started":1}
W20260810 18:48:58.177930 29563 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:58.172113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:33519 (local address 127.18.168.190:41605)
0810 18:48:58.172265 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:58.172269 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:58.172459 (+ 190us) server_negotiation.cc:407] Connection header received
0810 18:48:58.172575 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:58.172577 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:58.172622 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:58.172749 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:58.173223 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:58.174051 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:58.174891 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:58.175111 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:58.175194 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:58.177777 (+ 2583us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.130:33519: BlockingRecv error: recv got EOF from 127.18.168.130:33519 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 10 18:48:58 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.131@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:48:58 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.131@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:48:58.373767 29566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:58.368487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.131:41693)
0810 18:48:58.368840 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:58.368853 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:58.368947 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:58.369195 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:58.369198 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:58.369414 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:58.369530 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:58.369540 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:58.370291 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:58.370294 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:58.370844 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:58.370852 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:58.371294 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:58.371297 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:58.371361 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:58.371808 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:58.371820 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:58.373655 (+ 1835us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":96,"threads_started":1}
W20260810 18:48:58.373905 29563 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:58.368529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:41693 (local address 127.18.168.190:41605)
0810 18:48:58.368676 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:58.368681 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:58.368921 (+ 240us) server_negotiation.cc:407] Connection header received
0810 18:48:58.369025 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:58.369028 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:58.369074 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:58.369141 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:58.369696 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:58.370185 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:58.370979 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:58.371197 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:58.371275 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:58.373785 (+ 2510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.131:41693: BlockingRecv error: recv got EOF from 127.18.168.131:41693 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 10 18:48:58 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.129@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:48:58 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.129@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:48:58.995023 29567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:58.987652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.129:59957)
0810 18:48:58.987946 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:58.987960 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:58.988050 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:58.988324 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:58.988326 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:58.988515 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:58.988614 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:58.988619 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:58.989730 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:58.989734 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:58.990764 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:58.990779 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:58.991385 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:58.991393 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:58.991511 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:58.992215 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:58.992236 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:58.994821 (+ 2585us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":101,"threads_started":1}
W20260810 18:48:58.995169 29568 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:58.987719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:59957 (local address 127.18.168.190:41605)
0810 18:48:58.988019 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:58.988023 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:58.988039 (+ 16us) server_negotiation.cc:407] Connection header received
0810 18:48:58.988142 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:58.988144 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:58.988188 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:58.988282 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:58.988772 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:58.989566 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:58.990927 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:58.991234 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:58.991348 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:58.994996 (+ 3648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.129:59957: BlockingRecv error: recv got EOF from 127.18.168.129:59957 (error 108)
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":124,"threads_started":1}
Aug 10 18:48:59 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.130@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:48:59 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.130@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:48:59.184165 29569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:59.178564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.130:37055)
0810 18:48:59.178905 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:59.178921 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:59.179032 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:59.179324 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:59.179328 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:59.179550 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:59.179642 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:59.179649 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:59.180422 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:59.180428 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:59.181052 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:59.181061 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:59.181553 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:59.181557 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:59.181644 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:59.182168 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:59.182180 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:59.183983 (+ 1803us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":141,"threads_started":1}
W20260810 18:48:59.184206 29568 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:59.178739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:37055 (local address 127.18.168.190:41605)
0810 18:48:59.178899 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:59.178904 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:59.178979 (+ 75us) server_negotiation.cc:407] Connection header received
0810 18:48:59.179145 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:59.179149 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:59.179199 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:59.179297 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:59.179768 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:59.180310 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:59.181191 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:59.181434 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:59.181547 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:59.184108 (+ 2561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.130:37055: BlockingRecv error: recv got EOF from 127.18.168.130:37055 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 10 18:48:59 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.131@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:48:59 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.131@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:48:59.380025 29570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:59.374682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.131:49569)
0810 18:48:59.374941 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:59.374953 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:59.375049 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:59.375268 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:59.375271 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:59.375463 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:59.375557 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:59.375562 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:59.376444 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:59.376446 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:59.377016 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:59.377024 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:59.377495 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:59.377498 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:59.377559 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:59.378062 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:59.378073 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:59.379910 (+ 1837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":94,"threads_started":1}
W20260810 18:48:59.380178 29568 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:59.374766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:49569 (local address 127.18.168.190:41605)
0810 18:48:59.374906 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:59.374910 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:59.375006 (+ 96us) server_negotiation.cc:407] Connection header received
0810 18:48:59.375116 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:59.375118 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:59.375161 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:59.375230 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:59.375720 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:59.376339 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:59.377140 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:59.377389 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:59.377488 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:59.380062 (+ 2574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.131:49569: BlockingRecv error: recv got EOF from 127.18.168.131:49569 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 10 18:48:59 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.129@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:48:59 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.129@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:00.002224 29571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:59.995937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.129:59129)
0810 18:48:59.996252 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:59.996272 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:59.996431 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:59.996774 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:59.996776 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:59.996967 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:59.997083 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:59.997089 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:59.997868 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:59.997870 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:59.998506 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:59.998514 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:59.998936 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:59.998938 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:59.999005 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:59.999505 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:59.999517 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:00.002009 (+ 2492us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":86,"threads_started":1}
W20260810 18:49:00.002388 29572 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:59.996095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:59129 (local address 127.18.168.190:41605)
0810 18:48:59.996391 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:59.996396 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:59.996410 (+ 14us) server_negotiation.cc:407] Connection header received
0810 18:48:59.996543 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:59.996545 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:59.996587 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:59.996667 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:59.997220 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:59.997760 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:59.998613 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:59.998835 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:59.998911 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:00.002208 (+ 3297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.129:59129: BlockingRecv error: recv got EOF from 127.18.168.129:59129 (error 108)
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":85,"threads_started":1}
Aug 10 18:49:00 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.130@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:00 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.130@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:00.191008 29573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:00.185125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.130:46941)
0810 18:49:00.185516 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:00.185536 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:00.185670 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:00.185950 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:00.185955 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:00.186197 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:00.186319 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:00.186325 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:00.187097 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:00.187100 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:00.187708 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:00.187717 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:00.188133 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:00.188136 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:00.188204 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:00.188737 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:00.188750 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:00.190876 (+ 2126us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":134,"threads_started":1}
W20260810 18:49:00.191107 29572 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:00.185270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:46941 (local address 127.18.168.190:41605)
0810 18:49:00.185465 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:00.185469 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:00.185634 (+ 165us) server_negotiation.cc:407] Connection header received
0810 18:49:00.185760 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:00.185763 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:00.185809 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:00.185880 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:00.186428 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:00.186996 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:00.187833 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:00.188032 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:00.188105 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:00.191013 (+ 2908us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.130:46941: BlockingRecv error: recv got EOF from 127.18.168.130:46941 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 10 18:49:00 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.131@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:00 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.131@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:00.386252 29574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:00.380889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.131:52933)
0810 18:49:00.381189 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:00.381202 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:00.381303 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:00.381555 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:00.381558 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:00.381746 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:00.381842 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:00.381848 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:00.382579 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:00.382581 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:00.383129 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:00.383136 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:00.383603 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:00.383607 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:00.383698 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:00.384214 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:00.384225 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:00.386137 (+ 1912us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":121,"threads_started":1}
W20260810 18:49:00.386401 29572 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:00.381024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:52933 (local address 127.18.168.190:41605)
0810 18:49:00.381163 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:00.381167 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:00.381272 (+ 105us) server_negotiation.cc:407] Connection header received
0810 18:49:00.381382 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:00.381385 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:00.381432 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:00.381519 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:00.381939 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:00.382451 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:00.383235 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:00.383449 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:00.383557 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:00.386276 (+ 2719us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.131:52933: BlockingRecv error: recv got EOF from 127.18.168.131:52933 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 10 18:49:01 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.129@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:01 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.129@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:01.010102 29576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:01.003197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.129:59839)
0810 18:49:01.003757 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:01.003771 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:01.003880 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:01.004223 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:01.004227 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:01.004477 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:01.004634 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:01.004646 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:01.005928 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:01.005933 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:01.006643 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:01.006653 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:01.007448 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:01.007450 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:01.007519 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:01.008008 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:01.008020 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:01.009986 (+ 1966us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":164,"threads_started":1}
W20260810 18:49:01.010512 29575 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:01.003262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:59839 (local address 127.18.168.190:41605)
0810 18:49:01.003598 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:01.003603 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:01.003851 (+ 248us) server_negotiation.cc:407] Connection header received
0810 18:49:01.003990 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:01.003996 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:01.004051 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:01.004170 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:01.004816 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:01.005798 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:01.006764 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:01.007197 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:01.007308 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:01.010159 (+ 2851us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.129:59839: BlockingRecv error: recv got EOF from 127.18.168.129:59839 (error 108)
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":126,"threads_started":1}
Aug 10 18:49:01 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.130@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:01 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.130@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:01.197856 29577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:01.191909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.130:53041)
0810 18:49:01.192251 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:01.192264 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:01.192359 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:01.192721 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:01.192723 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:01.192910 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:01.193003 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:01.193008 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:01.193880 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:01.193885 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:01.194467 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:01.194476 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:01.195005 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:01.195008 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:01.195101 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:01.195645 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:01.195657 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:01.197697 (+ 2040us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":119,"threads_started":1}
W20260810 18:49:01.197960 29575 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:01.191991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:53041 (local address 127.18.168.190:41605)
0810 18:49:01.192134 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:01.192138 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:01.192326 (+ 188us) server_negotiation.cc:407] Connection header received
0810 18:49:01.192540 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:01.192543 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:01.192590 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:01.192663 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:01.193120 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:01.193752 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:01.194597 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:01.194889 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:01.194998 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:01.197846 (+ 2848us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.130:53041: BlockingRecv error: recv got EOF from 127.18.168.130:53041 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 10 18:49:01 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.131@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:01 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.131@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:01.392215 29578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:01.387036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.131:40837)
0810 18:49:01.387368 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:01.387382 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:01.387483 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:01.387709 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:01.387711 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:01.387899 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:01.387991 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:01.387996 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:01.388782 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:01.388784 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:01.389333 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:01.389341 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:01.389752 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:01.389754 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:01.389812 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:01.390295 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:01.390307 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:01.392094 (+ 1787us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":112,"threads_started":1}
W20260810 18:49:01.392347 29575 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:01.387108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:40837 (local address 127.18.168.190:41605)
0810 18:49:01.387272 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:01.387277 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:01.387449 (+ 172us) server_negotiation.cc:407] Connection header received
0810 18:49:01.387554 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:01.387556 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:01.387603 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:01.387671 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:01.388089 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:01.388628 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:01.389428 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:01.389660 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:01.389736 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:01.392227 (+ 2491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.131:40837: BlockingRecv error: recv got EOF from 127.18.168.131:40837 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 10 18:49:02 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.129@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:02 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.129@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:02.016876 29579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:02.010960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.129:40805)
0810 18:49:02.011250 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:02.011264 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:02.011353 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:02.011620 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:02.011622 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:02.011900 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:02.012033 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:02.012041 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:02.013074 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:02.013078 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:02.013658 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:02.013666 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:02.014072 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:02.014074 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:02.014135 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:02.014647 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:02.014663 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:02.016674 (+ 2011us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":100,"threads_started":1}
W20260810 18:49:02.016956 29580 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:02.011083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:40805 (local address 127.18.168.190:41605)
0810 18:49:02.011331 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:02.011334 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:02.011349 (+ 15us) server_negotiation.cc:407] Connection header received
0810 18:49:02.011436 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:02.011438 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:02.011482 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:02.011597 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:02.012154 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:02.012960 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:02.013756 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:02.013972 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:02.014056 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:02.016829 (+ 2773us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.129:40805: BlockingRecv error: recv got EOF from 127.18.168.129:40805 (error 108)
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":95,"threads_started":1}
Aug 10 18:49:02 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.130@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:02 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.130@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:02.206141 29581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:02.198902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.130:38107)
0810 18:49:02.199222 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:02.199236 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:02.199366 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:02.199645 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:02.199648 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:02.199863 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:02.199958 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:02.199964 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:02.201328 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:02.201331 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:02.202591 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:02.202604 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:02.203285 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:02.203292 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:02.203462 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:02.203962 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:02.203975 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:02.205975 (+ 2000us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":92,"threads_started":1}
W20260810 18:49:02.206220 29580 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:02.198992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:38107 (local address 127.18.168.190:41605)
0810 18:49:02.199168 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:02.199172 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:02.199307 (+ 135us) server_negotiation.cc:407] Connection header received
0810 18:49:02.199480 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:02.199483 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:02.199538 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:02.199619 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:02.200089 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:02.201208 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:02.202735 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:02.203118 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:02.203236 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:02.206097 (+ 2861us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.130:38107: BlockingRecv error: recv got EOF from 127.18.168.130:38107 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 10 18:49:02 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.131@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:02 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.131@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:02.398404 29582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:02.392968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.131:35719)
0810 18:49:02.393249 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:02.393263 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:02.393358 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:02.393571 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:02.393574 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:02.393756 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:02.393850 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:02.393855 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:02.394743 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:02.394745 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:02.395326 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:02.395334 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:02.395739 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:02.395741 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:02.395801 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:02.396277 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:02.396289 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:02.398235 (+ 1946us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":90,"threads_started":1}
W20260810 18:49:02.398465 29580 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:02.393013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:35719 (local address 127.18.168.190:41605)
0810 18:49:02.393152 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:02.393157 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:02.393326 (+ 169us) server_negotiation.cc:407] Connection header received
0810 18:49:02.393435 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:02.393437 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:02.393485 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:02.393552 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:02.393941 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:02.394616 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:02.395419 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:02.395621 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:02.395722 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:02.398356 (+ 2634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.131:35719: BlockingRecv error: recv got EOF from 127.18.168.131:35719 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 10 18:49:03 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.129@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:03 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.129@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:03.028868 29583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:03.017769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.129:36513)
0810 18:49:03.018088 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:03.018102 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:03.018228 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:03.018542 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:03.018545 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:03.018756 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:03.018851 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:03.018857 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:03.019648 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:03.019651 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:03.020202 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:03.020210 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:03.020735 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:03.020739 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:03.020832 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:03.026700 (+ 5868us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:03.026712 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:03.028752 (+ 2040us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":96,"threads_started":1}
W20260810 18:49:03.029094 29584 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:03.017865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:36513 (local address 127.18.168.190:41605)
0810 18:49:03.018162 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:03.018166 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:03.018180 (+ 14us) server_negotiation.cc:407] Connection header received
0810 18:49:03.018344 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:03.018347 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:03.018407 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:03.018515 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:03.018981 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:03.019541 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:03.020323 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:03.020583 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:03.020729 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:03.028901 (+ 8172us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.129:36513: BlockingRecv error: recv got EOF from 127.18.168.129:36513 (error 108)
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":91,"threads_started":1}
Aug 10 18:49:03 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.130@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:03 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.130@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:03.212334 29586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:03.207128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.130:45627)
0810 18:49:03.207388 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:03.207402 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:03.207505 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:03.207697 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:03.207699 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:03.207880 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:03.207975 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:03.207980 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:03.208720 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:03.208722 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:03.209266 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:03.209273 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:03.209706 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:03.209708 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:03.209773 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:03.210242 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:03.210253 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:03.212168 (+ 1915us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":77,"threads_started":1}
W20260810 18:49:03.212428 29584 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:03.207203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:45627 (local address 127.18.168.190:41605)
0810 18:49:03.207383 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:03.207388 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:03.207469 (+ 81us) server_negotiation.cc:407] Connection header received
0810 18:49:03.207573 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:03.207575 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:03.207620 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:03.207686 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:03.208066 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:03.208602 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:03.209369 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:03.209601 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:03.209680 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:03.212312 (+ 2632us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.130:45627: BlockingRecv error: recv got EOF from 127.18.168.130:45627 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 10 18:49:03 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.131@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:03 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.131@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:03.404492 29587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:03.399155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.131:39321)
0810 18:49:03.399447 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:03.399460 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:03.399567 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:03.399804 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:03.399806 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:03.399992 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:03.400093 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:03.400098 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:03.400837 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:03.400840 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:03.401430 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:03.401439 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:03.401939 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:03.401942 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:03.402004 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:03.402492 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:03.402504 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:03.404344 (+ 1840us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":103,"threads_started":1}
W20260810 18:49:03.404615 29584 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:03.399203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:39321 (local address 127.18.168.190:41605)
0810 18:49:03.399358 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:03.399362 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:03.399532 (+ 170us) server_negotiation.cc:407] Connection header received
0810 18:49:03.399642 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:03.399644 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:03.399699 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:03.399769 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:03.400193 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:03.400751 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:03.401529 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:03.401835 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:03.401918 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:03.404478 (+ 2560us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.131:39321: BlockingRecv error: recv got EOF from 127.18.168.131:39321 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 10 18:49:04 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.129@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:04 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.129@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:04.034700 29588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:04.029609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.129:44865)
0810 18:49:04.029887 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:04.029901 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:04.029989 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:04.030228 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:04.030230 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:04.030410 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:04.030509 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:04.030514 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:04.031214 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:04.031216 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:04.031754 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:04.031761 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:04.032170 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:04.032173 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:04.032238 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:04.032822 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:04.032834 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:04.034573 (+ 1739us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":87,"threads_started":1}
W20260810 18:49:04.034857 29589 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:04.029685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:44865 (local address 127.18.168.190:41605)
0810 18:49:04.029928 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:04.029931 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:04.029945 (+ 14us) server_negotiation.cc:407] Connection header received
0810 18:49:04.030076 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:04.030078 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:04.030119 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:04.030194 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:04.030612 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:04.031125 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:04.031847 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:04.032055 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:04.032146 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:04.034706 (+ 2560us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.129:44865: BlockingRecv error: recv got EOF from 127.18.168.129:44865 (error 108)
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":60,"threads_started":1}
Aug 10 18:49:04 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.130@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:04 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.130@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:04.218487 29590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:04.213147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.130:38917)
0810 18:49:04.213399 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:04.213414 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:04.213519 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:04.213748 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:04.213750 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:04.213927 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:04.214028 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:04.214033 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:04.214738 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:04.214740 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:04.215293 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:04.215301 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:04.215722 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:04.215724 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:04.215788 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:04.216348 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:04.216368 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:04.218306 (+ 1938us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":87,"threads_started":1}
W20260810 18:49:04.218573 29589 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:04.213189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:38917 (local address 127.18.168.190:41605)
0810 18:49:04.213336 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:04.213339 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:04.213485 (+ 146us) server_negotiation.cc:407] Connection header received
0810 18:49:04.213596 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:04.213598 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:04.213641 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:04.213710 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:04.214122 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:04.214645 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:04.215393 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:04.215622 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:04.215703 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:04.218436 (+ 2733us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.130:38917: BlockingRecv error: recv got EOF from 127.18.168.130:38917 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 10 18:49:04 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.131@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:04 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.131@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:04.410557 29591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:04.405263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.131:52203)
0810 18:49:04.405582 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:04.405597 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:04.405696 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:04.405957 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:04.405959 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:04.406152 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:04.406245 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:04.406250 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:04.406985 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:04.406987 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:04.407546 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:04.407553 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:04.407945 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:04.407947 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:04.408006 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:04.408503 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:04.408515 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:04.410426 (+ 1911us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":125,"threads_started":1}
W20260810 18:49:04.410679 29589 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:04.405609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:52203 (local address 127.18.168.190:41605)
0810 18:49:04.405752 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:04.405755 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:04.405767 (+ 12us) server_negotiation.cc:407] Connection header received
0810 18:49:04.405803 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:04.405805 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:04.405847 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:04.405913 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:04.406357 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:04.406891 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:04.407641 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:04.407852 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:04.407928 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:04.410557 (+ 2629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.131:52203: BlockingRecv error: recv got EOF from 127.18.168.131:52203 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 10 18:49:05 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.129@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:05 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.129@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:05.041241 29592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:05.035538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.129:47831)
0810 18:49:05.035855 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:05.035869 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:05.035974 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:05.036233 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:05.036235 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:05.036408 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:05.036508 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:05.036513 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:05.037477 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:05.037480 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:05.038281 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:05.038291 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:05.038761 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:05.038763 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:05.038823 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:05.039302 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:05.039314 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:05.041126 (+ 1812us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":67,"threads_started":1}
W20260810 18:49:05.041386 29593 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:05.035606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:47831 (local address 127.18.168.190:41605)
0810 18:49:05.035854 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:05.035858 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:05.035937 (+ 79us) server_negotiation.cc:407] Connection header received
0810 18:49:05.036054 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:05.036056 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:05.036101 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:05.036189 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:05.036638 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:05.037333 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:05.038427 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:05.038669 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:05.038748 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:05.041257 (+ 2509us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.129:47831: BlockingRecv error: recv got EOF from 127.18.168.129:47831 (error 108)
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":69,"threads_started":1}
Aug 10 18:49:05 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.130@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:05 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.130@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:05.225149 29594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:05.219395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.130:53291)
0810 18:49:05.219661 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:05.219675 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:05.219779 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:05.220043 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:05.220046 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:05.220311 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:05.220424 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:05.220430 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:05.221445 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:05.221448 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:05.222015 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:05.222033 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:05.222469 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:05.222472 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:05.222545 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:05.223053 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:05.223065 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:05.224986 (+ 1921us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":72,"threads_started":1}
W20260810 18:49:05.225217 29593 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:05.219512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:53291 (local address 127.18.168.190:41605)
0810 18:49:05.219682 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:05.219686 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:05.219727 (+ 41us) server_negotiation.cc:407] Connection header received
0810 18:49:05.219860 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:05.219863 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:05.219932 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:05.220020 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:05.220549 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:05.221305 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:05.222130 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:05.222361 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:05.222467 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:05.225126 (+ 2659us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.130:53291: BlockingRecv error: recv got EOF from 127.18.168.130:53291 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 10 18:49:05 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.131@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:05 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.131@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:05.417016 29595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:05.411483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.131:38357)
0810 18:49:05.411756 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:05.411771 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:05.411875 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:05.412143 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:05.412145 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:05.412321 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:05.412415 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:05.412420 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:05.413169 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:05.413171 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:05.413704 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:05.413711 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:05.414126 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:05.414129 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:05.414191 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:05.414745 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:05.414764 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:05.416848 (+ 2084us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":105,"threads_started":1}
W20260810 18:49:05.417081 29593 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:05.411551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:38357 (local address 127.18.168.190:41605)
0810 18:49:05.411690 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:05.411693 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:05.411842 (+ 149us) server_negotiation.cc:407] Connection header received
0810 18:49:05.411952 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:05.411954 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:05.411999 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:05.412114 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:05.412516 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:05.413080 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:05.413815 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:05.414035 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:05.414108 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:05.416963 (+ 2855us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.131:38357: BlockingRecv error: recv got EOF from 127.18.168.131:38357 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 10 18:49:06 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.129@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:06 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.129@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:06.047983 29596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:06.042175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.129:39121)
0810 18:49:06.042455 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:06.042468 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:06.042567 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:06.042780 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:06.042782 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:06.042964 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:06.043064 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:06.043069 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:06.044135 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:06.044142 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:06.044936 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:06.044949 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:06.045382 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:06.045385 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:06.045449 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:06.046006 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:06.046019 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:06.047868 (+ 1849us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":96,"threads_started":1}
W20260810 18:49:06.048202 29597 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:06.042240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:39121 (local address 127.18.168.190:41605)
0810 18:49:06.042482 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:06.042486 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:06.042517 (+ 31us) server_negotiation.cc:407] Connection header received
0810 18:49:06.042633 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:06.042635 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:06.042677 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:06.042755 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:06.043198 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:06.043994 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:06.045057 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:06.045284 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:06.045368 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:06.047999 (+ 2631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.129:39121: BlockingRecv error: recv got EOF from 127.18.168.129:39121 (error 108)
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":94,"threads_started":1}
Aug 10 18:49:06 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.130@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:06 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.130@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:06.231530 29598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:06.225889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.130:43165)
0810 18:49:06.226158 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:06.226174 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:06.226285 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:06.226530 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:06.226533 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:06.226744 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:06.226871 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:06.226880 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:06.227895 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:06.227898 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:06.228506 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:06.228516 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:06.229003 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:06.229005 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:06.229079 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:06.229563 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:06.229574 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:06.231407 (+ 1833us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":115,"threads_started":1}
W20260810 18:49:06.231729 29597 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:06.225968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:43165 (local address 127.18.168.190:41605)
0810 18:49:06.226120 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:06.226125 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:06.226254 (+ 129us) server_negotiation.cc:407] Connection header received
0810 18:49:06.226363 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:06.226370 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:06.226418 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:06.226492 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:06.227023 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:06.227769 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:06.228615 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:06.228891 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:06.228990 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:06.231563 (+ 2573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.130:43165: BlockingRecv error: recv got EOF from 127.18.168.130:43165 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 10 18:49:06 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.131@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:06 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.131@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:06.423585 29600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:06.417830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.131:53379)
0810 18:49:06.418198 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:06.418213 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:06.418350 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:06.418622 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:06.418625 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:06.418849 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:06.418936 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:06.418941 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:06.419692 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:06.419697 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:06.420232 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:06.420240 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:06.420680 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:06.420684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:06.420792 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:06.421395 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:06.421410 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:06.423470 (+ 2060us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":156,"threads_started":1}
W20260810 18:49:06.423836 29597 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:06.417985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:53379 (local address 127.18.168.190:41605)
0810 18:49:06.418174 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:06.418179 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:06.418295 (+ 116us) server_negotiation.cc:407] Connection header received
0810 18:49:06.418450 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:06.418454 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:06.418507 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:06.418599 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:06.419068 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:06.419588 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:06.420349 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:06.420573 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:06.420680 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:06.423611 (+ 2931us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.131:53379: BlockingRecv error: recv got EOF from 127.18.168.131:53379 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 10 18:49:07 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.129@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:07 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.129@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:07.054524 29601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:07.048946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.129:52675)
0810 18:49:07.049250 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:07.049264 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:07.049375 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:07.049639 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:07.049641 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:07.049826 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:07.049927 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:07.049933 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:07.050712 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:07.050714 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:07.051440 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:07.051452 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:07.052028 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:07.052032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:07.052112 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:07.052603 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:07.052615 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:07.054410 (+ 1795us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":112,"threads_started":1}
W20260810 18:49:07.054801 29602 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:07.049011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:52675 (local address 127.18.168.190:41605)
0810 18:49:07.049279 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:07.049283 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:07.049342 (+ 59us) server_negotiation.cc:407] Connection header received
0810 18:49:07.049454 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:07.049456 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:07.049499 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:07.049604 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:07.050033 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:07.050580 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:07.051566 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:07.051899 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:07.052005 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:07.054653 (+ 2648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.129:52675: BlockingRecv error: recv got EOF from 127.18.168.129:52675 (error 108)
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":95,"threads_started":1}
Aug 10 18:49:07 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.130@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:07 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.130@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:07.237820 29603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:07.232328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.130:43787)
0810 18:49:07.232635 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:07.232657 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:07.232819 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:07.233063 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:07.233066 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:07.233252 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:07.233348 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:07.233353 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:07.234077 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:07.234080 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:07.234616 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:07.234623 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:07.235176 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:07.235194 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:07.235290 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:07.235844 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:07.235856 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:07.237706 (+ 1850us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":132,"threads_started":1}
W20260810 18:49:07.237982 29602 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:07.232440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:43787 (local address 127.18.168.190:41605)
0810 18:49:07.232623 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:07.232627 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:07.232780 (+ 153us) server_negotiation.cc:407] Connection header received
0810 18:49:07.232905 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:07.232907 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:07.232952 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:07.233027 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:07.233445 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:07.233984 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:07.234709 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:07.235042 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:07.235164 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:07.237832 (+ 2668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.130:43787: BlockingRecv error: recv got EOF from 127.18.168.130:43787 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 10 18:49:07 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.131@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:07 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.131@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:07.430787 29604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:07.424318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.131:42611)
0810 18:49:07.424682 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:07.424731 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:07.424875 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:07.425223 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:07.425227 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:07.425490 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:07.425596 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:07.425603 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:07.426490 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:07.426497 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:07.427135 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:07.427144 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:07.427661 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:07.427666 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:07.427789 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:07.428474 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:07.428498 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:07.430665 (+ 2167us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":108,"threads_started":1}
W20260810 18:49:07.431012 29602 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:07.424432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:42611 (local address 127.18.168.190:41605)
0810 18:49:07.424621 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:07.424628 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:07.424810 (+ 182us) server_negotiation.cc:407] Connection header received
0810 18:49:07.425009 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:07.425014 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:07.425082 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:07.425196 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:07.425740 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:07.426348 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:07.427274 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:07.427533 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:07.427659 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:07.430865 (+ 3206us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.131:42611: BlockingRecv error: recv got EOF from 127.18.168.131:42611 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 10 18:49:08 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.129@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:08 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.129@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:08.068325 29605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:08.055401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.129:45123)
0810 18:49:08.055687 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:08.055700 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:08.055794 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:08.056045 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:08.056047 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:08.056234 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:08.056334 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:08.056340 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:08.057382 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:08.057385 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:08.058137 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:08.058156 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:08.058617 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:08.058621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:08.058722 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:08.066253 (+ 7531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:08.066265 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:08.068207 (+ 1942us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":96,"threads_started":1}
W20260810 18:49:08.068511 29606 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:08.055471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:45123 (local address 127.18.168.190:41605)
0810 18:49:08.055845 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:08.055849 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:08.055861 (+ 12us) server_negotiation.cc:407] Connection header received
0810 18:49:08.055897 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:08.055899 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:08.055939 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:08.056014 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:08.056433 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:08.057254 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:08.058253 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:08.058492 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:08.058576 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:08.068338 (+ 9762us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.129:45123: BlockingRecv error: recv got EOF from 127.18.168.129:45123 (error 108)
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":113,"threads_started":1}
Aug 10 18:49:08 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.130@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:08 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.130@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:08.243820 29608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:08.238518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.130:38023)
0810 18:49:08.238785 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:08.238800 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:08.238909 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:08.239156 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:08.239159 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:08.239342 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:08.239440 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:08.239446 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:08.240178 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:08.240180 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:08.240773 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:08.240781 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:08.241190 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:08.241193 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:08.241254 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:08.241728 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:08.241739 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:08.243669 (+ 1930us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":112,"threads_started":1}
W20260810 18:49:08.244140 29606 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:08.238596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:38023 (local address 127.18.168.190:41605)
0810 18:49:08.238737 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:08.238741 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:08.238876 (+ 135us) server_negotiation.cc:407] Connection header received
0810 18:49:08.238977 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:08.238979 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:08.239024 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:08.239092 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:08.239548 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:08.240079 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:08.240865 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:08.241085 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:08.241165 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:08.243862 (+ 2697us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.130:38023: BlockingRecv error: recv got EOF from 127.18.168.130:38023 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 10 18:49:08 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.131@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:08 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.131@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:08.437001 29609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:08.431506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.131:47567)
0810 18:49:08.431836 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:08.431849 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:08.431975 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:08.432236 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:08.432239 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:08.432453 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:08.432545 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:08.432550 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:08.433383 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:08.433389 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:08.433959 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:08.433967 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:08.434434 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:08.434437 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:08.434529 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:08.435013 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:08.435024 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:08.436817 (+ 1793us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":116,"threads_started":1}
W20260810 18:49:08.437040 29606 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:08.431574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:47567 (local address 127.18.168.190:41605)
0810 18:49:08.431742 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:08.431747 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:08.431920 (+ 173us) server_negotiation.cc:407] Connection header received
0810 18:49:08.432073 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:08.432077 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:08.432131 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:08.432216 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:08.432671 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:08.433273 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:08.434085 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:08.434321 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:08.434429 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:08.436934 (+ 2505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.131:47567: BlockingRecv error: recv got EOF from 127.18.168.131:47567 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 10 18:49:09 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.129@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:09 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.129@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:09.075111 29610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:09.069285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.129:54381)
0810 18:49:09.069589 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:09.069606 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:09.069712 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:09.069988 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:09.069991 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:09.070188 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:09.070299 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:09.070304 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:09.071223 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:09.071228 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:09.072101 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:09.072113 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:09.072547 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:09.072549 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:09.072611 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:09.073115 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:09.073126 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:09.074969 (+ 1843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":93,"threads_started":1}
W20260810 18:49:09.075311 29612 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:09.069325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:54381 (local address 127.18.168.190:41605)
0810 18:49:09.069628 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:09.069632 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:09.069667 (+ 35us) server_negotiation.cc:407] Connection header received
0810 18:49:09.069793 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:09.069795 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:09.069840 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:09.069922 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:09.070392 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:09.071110 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:09.072232 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:09.072453 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:09.072535 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:09.075113 (+ 2578us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.129:54381: BlockingRecv error: recv got EOF from 127.18.168.129:54381 (error 108)
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":85,"threads_started":1}
Aug 10 18:49:09 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.130@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:09 dist-test-slave-cbsf krb5kdc[28380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.168.130@KRBTEST.COM for oryx/127.18.168.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:09.250324 29613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:09.244752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41605 (local address 127.18.168.130:53551)
0810 18:49:09.245016 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:09.245032 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:09.245162 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:09.245437 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:09.245440 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:09.245660 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:09.245755 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:09.245760 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:09.246574 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:09.246577 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:09.247153 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:09.247162 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:09.247625 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:09.247628 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:09.247723 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:09.248265 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:09.248280 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:09.250155 (+ 1875us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.168.190:41605: Server oryx/127.18.168.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":86,"threads_started":1}
W20260810 18:49:09.250391 29612 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:09.244776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:53551 (local address 127.18.168.190:41605)
0810 18:49:09.244932 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:09.244937 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:09.245113 (+ 176us) server_negotiation.cc:407] Connection header received
0810 18:49:09.245290 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:09.245293 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:09.245336 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:09.245414 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:09.245883 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:09.246440 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:09.247279 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:09.247507 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:09.247620 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:09.250280 (+ 2660us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.168.130:53551: BlockingRecv error: recv got EOF from 127.18.168.130:53551 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 18:49:09.344642 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 29062
I20260810 18:49:09.355137 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 29240
I20260810 18:49:09.363233 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 29387
I20260810 18:49:09.370916 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 28982
2026-08-10T18:49:09Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18171 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-10T18:49:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:49:09Z Disabled control of system clock
I20260810 18:49:09.404250 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:40237
--webserver_interface=127.18.168.190
--webserver_port=0
--builtin_ntp_servers=127.18.168.148:44855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:40237
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 18:49:09.517709 29617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:09.517962 29617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:09.518000 29617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:09.522037 29617 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:49:09.522089 29617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:09.522104 29617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:09.522120 29617 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:49:09.522135 29617 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:49:09.526795 29617 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:44855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:40237
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:40237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:49:09.527787 29617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:09.529136 29617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 18:49:09.537708 29617 server_base.cc:1034] running on GCE node
W20260810 18:49:09.537735 29624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:09.538209 29626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:09.538216 29623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:09.538854 29617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:09.539897 29617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:09.541076 29617 hybrid_clock.cc:648] HybridClock initialized: now 1786387749541063 us; error 24 us; skew 500 ppm
I20260810 18:49:09.542877 29617 webserver.cc:460] Webserver started at http://127.18.168.190:36235/ using document root <none> and password file <none>
I20260810 18:49:09.543454 29617 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:09.543565 29617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:09.543792 29617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:09.545861 29617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "0764bcaac23e4ad3a4f72bcf4655b2d7"
format_stamp: "Formatted at 2026-08-10 18:49:09 on dist-test-slave-cbsf"
server_key: "31c608aa938282519f2e263e7e0d0ea1"
server_key_iv: "1771bb1f6868a128f010add5bcd5df09"
server_key_version: "encryptionkey@0"
I20260810 18:49:09.546420 29617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "0764bcaac23e4ad3a4f72bcf4655b2d7"
format_stamp: "Formatted at 2026-08-10 18:49:09 on dist-test-slave-cbsf"
server_key: "31c608aa938282519f2e263e7e0d0ea1"
server_key_iv: "1771bb1f6868a128f010add5bcd5df09"
server_key_version: "encryptionkey@0"
I20260810 18:49:09.550069 29617 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:49:09.553153 29633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:09.554867 29617 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 18:49:09.555006 29617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "0764bcaac23e4ad3a4f72bcf4655b2d7"
format_stamp: "Formatted at 2026-08-10 18:49:09 on dist-test-slave-cbsf"
server_key: "31c608aa938282519f2e263e7e0d0ea1"
server_key_iv: "1771bb1f6868a128f010add5bcd5df09"
server_key_version: "encryptionkey@0"
I20260810 18:49:09.555156 29617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:09.588641 29617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:09.589654 29617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:09.589895 29617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:09.598892 29617 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:40237
I20260810 18:49:09.598969 29692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:40237 every 8 connection(s)
I20260810 18:49:09.600023 29617 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:49:09.600620 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 29617
I20260810 18:49:09.600801 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:49:09.601118 19106 external_mini_cluster.cc:1428] Setting key 1bec2280b9a8a87bb5040c145427248b
I20260810 18:49:09.604589 29693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:09.612793 29693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0764bcaac23e4ad3a4f72bcf4655b2d7: Bootstrap starting.
I20260810 18:49:09.612951 29696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:09.602567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54000 (local address 127.18.168.190:40237)
0810 18:49:09.603275 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:09.603295 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:09.603340 (+ 45us) server_negotiation.cc:407] Connection header received
0810 18:49:09.604339 (+ 999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:09.604365 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:09.604911 (+ 546us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:09.605296 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:09.605808 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:09.607299 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:09.608631 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:09.609413 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:09.609549 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:09.610768 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:09.610776 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:09.610793 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:49:09.611327 (+ 534us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:09.611406 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:49:09.611437 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:09.611553 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:09.611693 (+ 140us) server_negotiation.cc:299] Negotiation successful
0810 18:49:09.611983 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":147,"threads_started":1}
I20260810 18:49:09.615059 29693 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0764bcaac23e4ad3a4f72bcf4655b2d7: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:09.615803 29693 log.cc:824] T 00000000000000000000000000000000 P 0764bcaac23e4ad3a4f72bcf4655b2d7: Log is configured to *not* fsync() on all Append() calls
I20260810 18:49:09.617825 29693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0764bcaac23e4ad3a4f72bcf4655b2d7: No bootstrap required, opened a new log
I20260810 18:49:09.621184 29693 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0764bcaac23e4ad3a4f72bcf4655b2d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0764bcaac23e4ad3a4f72bcf4655b2d7" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 40237 } }
I20260810 18:49:09.621388 29693 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0764bcaac23e4ad3a4f72bcf4655b2d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:09.621423 29693 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0764bcaac23e4ad3a4f72bcf4655b2d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0764bcaac23e4ad3a4f72bcf4655b2d7, State: Initialized, Role: FOLLOWER
I20260810 18:49:09.621768 29693 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0764bcaac23e4ad3a4f72bcf4655b2d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0764bcaac23e4ad3a4f72bcf4655b2d7" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 40237 } }
I20260810 18:49:09.621886 29693 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0764bcaac23e4ad3a4f72bcf4655b2d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:49:09.621932 29693 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0764bcaac23e4ad3a4f72bcf4655b2d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:49:09.622000 29693 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0764bcaac23e4ad3a4f72bcf4655b2d7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:09.623241 29693 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0764bcaac23e4ad3a4f72bcf4655b2d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0764bcaac23e4ad3a4f72bcf4655b2d7" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 40237 } }
I20260810 18:49:09.623554 29693 leader_election.cc:302] T 00000000000000000000000000000000 P 0764bcaac23e4ad3a4f72bcf4655b2d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0764bcaac23e4ad3a4f72bcf4655b2d7; no voters:
I20260810 18:49:09.623811 29693 leader_election.cc:288] T 00000000000000000000000000000000 P 0764bcaac23e4ad3a4f72bcf4655b2d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:49:09.623944 29700 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0764bcaac23e4ad3a4f72bcf4655b2d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:49:09.624249 29700 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0764bcaac23e4ad3a4f72bcf4655b2d7 [term 1 LEADER]: Becoming Leader. State: Replica: 0764bcaac23e4ad3a4f72bcf4655b2d7, State: Running, Role: LEADER
I20260810 18:49:09.624584 29700 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0764bcaac23e4ad3a4f72bcf4655b2d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0764bcaac23e4ad3a4f72bcf4655b2d7" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 40237 } }
I20260810 18:49:09.625056 29693 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0764bcaac23e4ad3a4f72bcf4655b2d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:49:09.626354 29702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0764bcaac23e4ad3a4f72bcf4655b2d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0764bcaac23e4ad3a4f72bcf4655b2d7. Latest consensus state: current_term: 1 leader_uuid: "0764bcaac23e4ad3a4f72bcf4655b2d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0764bcaac23e4ad3a4f72bcf4655b2d7" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 40237 } } }
I20260810 18:49:09.626382 29701 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0764bcaac23e4ad3a4f72bcf4655b2d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0764bcaac23e4ad3a4f72bcf4655b2d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0764bcaac23e4ad3a4f72bcf4655b2d7" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 40237 } } }
I20260810 18:49:09.626577 29702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0764bcaac23e4ad3a4f72bcf4655b2d7 [sys.catalog]: This master's current role is: LEADER
I20260810 18:49:09.626601 29701 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0764bcaac23e4ad3a4f72bcf4655b2d7 [sys.catalog]: This master's current role is: LEADER
I20260810 18:49:09.627482 29710 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:49:09.630717 29710 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:49:09.636375 29710 catalog_manager.cc:1347] Generated new cluster ID: 08143a6af4314b9eab78f370391da54f
I20260810 18:49:09.636516 29710 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:49:09.649142 29710 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:49:09.650354 29710 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:49:09.657007 29710 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0764bcaac23e4ad3a4f72bcf4655b2d7: Generated new TSK 0
I20260810 18:49:09.657826 29710 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:49:09.666038 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.129:0
--local_ip_for_outbound_sockets=127.18.168.129
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:40237
--builtin_ntp_servers=127.18.168.148:44855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260810 18:49:09.780443 29721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:09.780714 29721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:09.780750 29721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:09.784685 29721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:09.784786 29721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:09.784840 29721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.129
I20260810 18:49:09.789719 29721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:44855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:40237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.129
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:49:09.790810 29721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:09.792204 29721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:09.801854 29731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:09.801860 29728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:09.801874 29727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:09.802095 29721 server_base.cc:1034] running on GCE node
I20260810 18:49:09.802572 29721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:09.803249 29721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:09.804410 29721 hybrid_clock.cc:648] HybridClock initialized: now 1786387749804396 us; error 25 us; skew 500 ppm
I20260810 18:49:09.806357 29721 webserver.cc:460] Webserver started at http://127.18.168.129:38441/ using document root <none> and password file <none>
I20260810 18:49:09.806962 29721 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:09.807044 29721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:09.807289 29721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:09.809454 29721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/data/instance:
uuid: "5f81f545e6f64caca5cb45e3ff776685"
format_stamp: "Formatted at 2026-08-10 18:49:09 on dist-test-slave-cbsf"
server_key: "ced7646149ca57162050796d434b6e71"
server_key_iv: "da3e71b6282719fab1b0b97888a7ce6a"
server_key_version: "encryptionkey@0"
I20260810 18:49:09.810016 29721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance:
uuid: "5f81f545e6f64caca5cb45e3ff776685"
format_stamp: "Formatted at 2026-08-10 18:49:09 on dist-test-slave-cbsf"
server_key: "ced7646149ca57162050796d434b6e71"
server_key_iv: "da3e71b6282719fab1b0b97888a7ce6a"
server_key_version: "encryptionkey@0"
I20260810 18:49:09.813716 29721 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 18:49:09.816318 29738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:09.817575 29721 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:49:09.817706 29721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/wal
uuid: "5f81f545e6f64caca5cb45e3ff776685"
format_stamp: "Formatted at 2026-08-10 18:49:09 on dist-test-slave-cbsf"
server_key: "ced7646149ca57162050796d434b6e71"
server_key_iv: "da3e71b6282719fab1b0b97888a7ce6a"
server_key_version: "encryptionkey@0"
I20260810 18:49:09.817816 29721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:09.837121 29721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:09.837877 29721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:09.838075 29721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:09.838681 29721 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:49:09.839653 29721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:49:09.839721 29721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:09.839790 29721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:49:09.839836 29721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:09.848740 29721 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.129:38343
I20260810 18:49:09.848802 29867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.129:38343 every 8 connection(s)
I20260810 18:49:09.849772 29721 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
I20260810 18:49:09.853781 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 29721
I20260810 18:49:09.853909 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance
I20260810 18:49:09.854208 19106 external_mini_cluster.cc:1428] Setting key e4fd4e4b63e07d3c0a7a53476961445b
I20260810 18:49:09.856388 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.130:0
--local_ip_for_outbound_sockets=127.18.168.130
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:40237
--builtin_ntp_servers=127.18.168.148:44855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 18:49:09.860313 29696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:09.851663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:57097 (local address 127.18.168.190:40237)
0810 18:49:09.851805 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:09.851809 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:09.852953 (+ 1144us) server_negotiation.cc:407] Connection header received
0810 18:49:09.854851 (+ 1898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:09.854854 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:09.854910 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:09.855009 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:09.856136 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:09.856956 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:09.858192 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:09.858410 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:09.858494 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:09.859802 (+ 1308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:09.859805 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:09.859807 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:49:09.859820 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:09.859839 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:49:09.859846 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:09.859901 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:09.860087 (+ 186us) server_negotiation.cc:299] Negotiation successful
0810 18:49:09.860159 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 18:49:09.861582 29870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:09.852068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40237 (local address 127.18.168.129:57097)
0810 18:49:09.852788 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:09.852829 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:09.854601 (+ 1772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:09.855224 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:09.855232 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:09.855336 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:09.855986 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:09.856002 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:09.857089 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:09.857092 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:09.858037 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:09.858052 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:09.858571 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:09.858577 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:09.858807 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:09.859480 (+ 673us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:49:09.859483 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:09.859540 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:49:09.859630 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:09.859926 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:09.859938 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:09.859947 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 18:49:09.860083 (+ 136us) client_negotiation.cc:240] Negotiation successful
0810 18:49:09.860406 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":129,"threads_started":1}
I20260810 18:49:09.862845 29868 heartbeater.cc:347] Connected to a master server at 127.18.168.190:40237
I20260810 18:49:09.863126 29868 heartbeater.cc:464] Registering TS with master...
I20260810 18:49:09.863844 29868 heartbeater.cc:511] Master 127.18.168.190:40237 requested a full tablet report, sending...
I20260810 18:49:09.865509 29652 ts_manager.cc:194] Registered new tserver with Master: 5f81f545e6f64caca5cb45e3ff776685 (127.18.168.129:38343)
I20260810 18:49:09.866684 29652 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.168.129:57097
W20260810 18:49:09.970562 29871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:09.970793 29871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:09.970826 29871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:09.974756 29871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:09.974807 29871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:09.974887 29871 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.130
I20260810 18:49:09.979465 29871 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:44855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:40237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.130
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:49:09.980520 29871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:09.981886 29871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:09.991621 29877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:09.991616 29876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:09.991847 29871 server_base.cc:1034] running on GCE node
W20260810 18:49:09.991616 29879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:09.992326 29871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:09.993011 29871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:09.994196 29871 hybrid_clock.cc:648] HybridClock initialized: now 1786387749994180 us; error 27 us; skew 500 ppm
I20260810 18:49:09.995934 29871 webserver.cc:460] Webserver started at http://127.18.168.130:43809/ using document root <none> and password file <none>
I20260810 18:49:09.996471 29871 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:09.996575 29871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:09.996855 29871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:09.998920 29871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/data/instance:
uuid: "fa29cd4b9b144a2d9a556476f8ba5d37"
format_stamp: "Formatted at 2026-08-10 18:49:09 on dist-test-slave-cbsf"
server_key: "ef5651ec9d38a3ff45a1d0d43f6212f3"
server_key_iv: "fa6ad4265bf602bd4fd5a08e992a18b7"
server_key_version: "encryptionkey@0"
I20260810 18:49:09.999472 29871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance:
uuid: "fa29cd4b9b144a2d9a556476f8ba5d37"
format_stamp: "Formatted at 2026-08-10 18:49:09 on dist-test-slave-cbsf"
server_key: "ef5651ec9d38a3ff45a1d0d43f6212f3"
server_key_iv: "fa6ad4265bf602bd4fd5a08e992a18b7"
server_key_version: "encryptionkey@0"
I20260810 18:49:10.003049 29871 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 18:49:10.005558 29887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:10.006774 29871 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:49:10.006898 29871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/wal
uuid: "fa29cd4b9b144a2d9a556476f8ba5d37"
format_stamp: "Formatted at 2026-08-10 18:49:09 on dist-test-slave-cbsf"
server_key: "ef5651ec9d38a3ff45a1d0d43f6212f3"
server_key_iv: "fa6ad4265bf602bd4fd5a08e992a18b7"
server_key_version: "encryptionkey@0"
I20260810 18:49:10.007004 29871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:10.027037 29871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:10.027920 29871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:10.028141 29871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:10.028884 29871 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:49:10.029834 29871 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:49:10.029902 29871 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:10.029968 29871 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:49:10.029997 29871 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:10.038558 29871 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.130:33689
I20260810 18:49:10.038590 30016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.130:33689 every 8 connection(s)
I20260810 18:49:10.039449 29871 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
I20260810 18:49:10.045245 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 29871
I20260810 18:49:10.045377 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance
I20260810 18:49:10.045668 19106 external_mini_cluster.cc:1428] Setting key c57c7bc6b71289d56f8bfafe154838d9
I20260810 18:49:10.047652 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.131:0
--local_ip_for_outbound_sockets=127.18.168.131
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:40237
--builtin_ntp_servers=127.18.168.148:44855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 18:49:10.048962 29696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:10.041264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:32885 (local address 127.18.168.190:40237)
0810 18:49:10.041433 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:10.041438 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:10.042484 (+ 1046us) server_negotiation.cc:407] Connection header received
0810 18:49:10.043870 (+ 1386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:10.043873 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:10.043923 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:10.044001 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:10.044939 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:10.045625 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:10.046897 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:10.047144 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:10.047253 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:10.048454 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:10.048458 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:10.048460 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:49:10.048471 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:10.048491 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:49:10.048499 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:10.048578 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:10.048753 (+ 175us) server_negotiation.cc:299] Negotiation successful
0810 18:49:10.048819 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 18:49:10.049841 30019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:10.041695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40237 (local address 127.18.168.130:32885)
0810 18:49:10.042341 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:10.042382 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:10.043667 (+ 1285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:10.044166 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:10.044172 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:10.044252 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:10.044788 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:10.044803 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:10.045764 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:10.045769 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:10.046799 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:10.046810 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:10.047305 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:10.047310 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:10.047486 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:10.048173 (+ 687us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:49:10.048176 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:10.048233 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:49:10.048298 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:10.048580 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:10.048593 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:10.048600 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 18:49:10.048752 (+ 152us) client_negotiation.cc:240] Negotiation successful
0810 18:49:10.049037 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":101,"threads_started":1}
I20260810 18:49:10.050997 30017 heartbeater.cc:347] Connected to a master server at 127.18.168.190:40237
I20260810 18:49:10.051281 30017 heartbeater.cc:464] Registering TS with master...
I20260810 18:49:10.051817 30017 heartbeater.cc:511] Master 127.18.168.190:40237 requested a full tablet report, sending...
I20260810 18:49:10.053196 29652 ts_manager.cc:194] Registered new tserver with Master: fa29cd4b9b144a2d9a556476f8ba5d37 (127.18.168.130:33689)
I20260810 18:49:10.054214 29652 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.168.130:32885
W20260810 18:49:10.165555 30022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:10.165802 30022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:10.165838 30022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:10.169552 30022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:10.169600 30022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:10.169648 30022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.131
I20260810 18:49:10.174156 30022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:44855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:40237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.131
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:49:10.175133 30022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:10.176461 30022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:10.185870 30028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:10.185895 30034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:10.185953 30029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:10.186197 30022 server_base.cc:1034] running on GCE node
I20260810 18:49:10.186633 30022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:10.187315 30022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:10.188513 30022 hybrid_clock.cc:648] HybridClock initialized: now 1786387750188482 us; error 48 us; skew 500 ppm
I20260810 18:49:10.190449 30022 webserver.cc:460] Webserver started at http://127.18.168.131:36871/ using document root <none> and password file <none>
I20260810 18:49:10.191030 30022 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:10.191116 30022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:10.191334 30022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:10.193392 30022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/data/instance:
uuid: "dc2b6277594247688e4bbd4c217e69a6"
format_stamp: "Formatted at 2026-08-10 18:49:10 on dist-test-slave-cbsf"
server_key: "9cbbd544ea17b933e0acdb83b351a8c8"
server_key_iv: "f398460be31db5a3c9cad4f8b9dcc4b8"
server_key_version: "encryptionkey@0"
I20260810 18:49:10.193913 30022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance:
uuid: "dc2b6277594247688e4bbd4c217e69a6"
format_stamp: "Formatted at 2026-08-10 18:49:10 on dist-test-slave-cbsf"
server_key: "9cbbd544ea17b933e0acdb83b351a8c8"
server_key_iv: "f398460be31db5a3c9cad4f8b9dcc4b8"
server_key_version: "encryptionkey@0"
I20260810 18:49:10.197507 30022 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:49:10.199899 30041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:10.201100 30022 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:49:10.201236 30022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/wal
uuid: "dc2b6277594247688e4bbd4c217e69a6"
format_stamp: "Formatted at 2026-08-10 18:49:10 on dist-test-slave-cbsf"
server_key: "9cbbd544ea17b933e0acdb83b351a8c8"
server_key_iv: "f398460be31db5a3c9cad4f8b9dcc4b8"
server_key_version: "encryptionkey@0"
I20260810 18:49:10.201336 30022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:10.231153 30022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:10.231890 30022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:10.232074 30022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:10.232748 30022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:49:10.233773 30022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:49:10.233829 30022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:10.233875 30022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:49:10.233901 30022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:10.243172 30022 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.131:37347
I20260810 18:49:10.243197 30169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.131:37347 every 8 connection(s)
I20260810 18:49:10.244107 30022 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
I20260810 18:49:10.246078 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 30022
I20260810 18:49:10.246183 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance
I20260810 18:49:10.246476 19106 external_mini_cluster.cc:1428] Setting key b691ff6ec03d9319ca86f1a9997b82e2
I20260810 18:49:10.253293 29696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:10.246098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:51961 (local address 127.18.168.190:40237)
0810 18:49:10.246245 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:10.246250 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:10.247424 (+ 1174us) server_negotiation.cc:407] Connection header received
0810 18:49:10.248947 (+ 1523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:10.248949 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:10.248995 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:10.249062 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:10.249902 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:10.250418 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:10.251516 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:10.251735 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:10.251810 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:10.252892 (+ 1082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:10.252895 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:10.252897 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:49:10.252908 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:10.252929 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:49:10.252936 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:10.252995 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:10.253133 (+ 138us) server_negotiation.cc:299] Negotiation successful
0810 18:49:10.253186 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 18:49:10.254104 30172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:10.246484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40237 (local address 127.18.168.131:51961)
0810 18:49:10.247284 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:10.247335 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:10.248762 (+ 1427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:10.249223 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:10.249230 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:10.249311 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:10.249789 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:10.249801 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:10.250516 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:10.250518 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:10.251415 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:10.251425 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:10.251858 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:10.251860 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:10.251997 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:10.252572 (+ 575us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:49:10.252574 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:10.252612 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:49:10.252739 (+ 127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:10.253007 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:10.253013 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:10.253017 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 18:49:10.253120 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 18:49:10.253331 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":143,"threads_started":1}
I20260810 18:49:10.255185 30170 heartbeater.cc:347] Connected to a master server at 127.18.168.190:40237
I20260810 18:49:10.255429 30170 heartbeater.cc:464] Registering TS with master...
I20260810 18:49:10.255945 30170 heartbeater.cc:511] Master 127.18.168.190:40237 requested a full tablet report, sending...
I20260810 18:49:10.256954 29652 ts_manager.cc:194] Registered new tserver with Master: dc2b6277594247688e4bbd4c217e69a6 (127.18.168.131:37347)
I20260810 18:49:10.257550 29652 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.168.131:51961
I20260810 18:49:10.260289 19106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 18:49:10.263265 29696 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:10.262267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54006 (local address 127.18.168.190:40237)
0810 18:49:10.262409 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:10.262413 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:10.262604 (+ 191us) server_negotiation.cc:407] Connection header received
0810 18:49:10.262750 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:10.262753 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:10.262795 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:10.262878 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:10.263144 (+ 266us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54006: BlockingRecv error: recv got EOF from 127.0.0.1:54006 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 18:49:10.263391 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 29721
I20260810 18:49:10.274035 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 29871
I20260810 18:49:10.286473 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 30022
I20260810 18:49:10.295178 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 29617
2026-08-10T18:49:10Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (922 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-10T18:49:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:49:10Z Disabled control of system clock
I20260810 18:49:10.327508 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:36391
--webserver_interface=127.18.168.190
--webserver_port=0
--builtin_ntp_servers=127.18.168.148:41815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:36391
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 18:49:10.437803 30184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:10.438038 30184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:10.438073 30184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:10.441855 30184 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:49:10.441901 30184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:10.441916 30184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:10.441932 30184 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:49:10.441973 30184 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:49:10.446638 30184 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:41815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:36391
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:36391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30184
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:49:10.447589 30184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:10.448897 30184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:10.457219 30192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:10.457228 30189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:10.457259 30190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:10.457284 30184 server_base.cc:1034] running on GCE node
I20260810 18:49:10.457967 30184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:10.459064 30184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:10.460223 30184 hybrid_clock.cc:648] HybridClock initialized: now 1786387750460209 us; error 25 us; skew 500 ppm
I20260810 18:49:10.462014 30184 webserver.cc:460] Webserver started at http://127.18.168.190:40659/ using document root <none> and password file <none>
I20260810 18:49:10.462558 30184 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:10.462632 30184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:10.462872 30184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:10.464903 30184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "e078a58a1b894aeeb8c58da9aa6aa920"
format_stamp: "Formatted at 2026-08-10 18:49:10 on dist-test-slave-cbsf"
server_key: "1ebbe2c5b3d4ce73474bdc4c446c802d"
server_key_iv: "e3e3e58b71c9675b889c8acfdb69843d"
server_key_version: "encryptionkey@0"
I20260810 18:49:10.465476 30184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "e078a58a1b894aeeb8c58da9aa6aa920"
format_stamp: "Formatted at 2026-08-10 18:49:10 on dist-test-slave-cbsf"
server_key: "1ebbe2c5b3d4ce73474bdc4c446c802d"
server_key_iv: "e3e3e58b71c9675b889c8acfdb69843d"
server_key_version: "encryptionkey@0"
I20260810 18:49:10.468962 30184 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 18:49:10.471202 30198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:10.472332 30184 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:49:10.472465 30184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "e078a58a1b894aeeb8c58da9aa6aa920"
format_stamp: "Formatted at 2026-08-10 18:49:10 on dist-test-slave-cbsf"
server_key: "1ebbe2c5b3d4ce73474bdc4c446c802d"
server_key_iv: "e3e3e58b71c9675b889c8acfdb69843d"
server_key_version: "encryptionkey@0"
I20260810 18:49:10.472591 30184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:10.477063 30184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:10.477663 30184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:10.477837 30184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:10.484624 30184 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:36391
I20260810 18:49:10.484654 30254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:36391 every 8 connection(s)
I20260810 18:49:10.485579 30184 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:49:10.488790 30256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:10.492892 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 30184
I20260810 18:49:10.493012 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:49:10.493307 19106 external_mini_cluster.cc:1428] Setting key 3491c8ef99fee4596d61f6666e46aa07
I20260810 18:49:10.494683 30256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e078a58a1b894aeeb8c58da9aa6aa920: Bootstrap starting.
I20260810 18:49:10.496950 30256 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e078a58a1b894aeeb8c58da9aa6aa920: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:10.497741 30256 log.cc:824] T 00000000000000000000000000000000 P e078a58a1b894aeeb8c58da9aa6aa920: Log is configured to *not* fsync() on all Append() calls
I20260810 18:49:10.498658 30259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:10.494694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60792 (local address 127.18.168.190:36391)
0810 18:49:10.495306 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:10.495321 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:10.495355 (+ 34us) server_negotiation.cc:407] Connection header received
0810 18:49:10.495965 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:10.495981 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:10.496302 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:10.496643 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:10.497419 (+ 776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:10.497423 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:10.497434 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:49:10.497445 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:10.497503 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:49:10.497518 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:10.497642 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:10.497744 (+ 102us) server_negotiation.cc:299] Negotiation successful
0810 18:49:10.497986 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":130,"threads_started":1}
I20260810 18:49:10.499732 30256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e078a58a1b894aeeb8c58da9aa6aa920: No bootstrap required, opened a new log
I20260810 18:49:10.503343 30256 raft_consensus.cc:348] T 00000000000000000000000000000000 P e078a58a1b894aeeb8c58da9aa6aa920 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e078a58a1b894aeeb8c58da9aa6aa920" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 36391 } }
I20260810 18:49:10.503543 30256 raft_consensus.cc:374] T 00000000000000000000000000000000 P e078a58a1b894aeeb8c58da9aa6aa920 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:10.503624 30256 raft_consensus.cc:729] T 00000000000000000000000000000000 P e078a58a1b894aeeb8c58da9aa6aa920 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e078a58a1b894aeeb8c58da9aa6aa920, State: Initialized, Role: FOLLOWER
I20260810 18:49:10.504093 30256 consensus_queue.cc:260] T 00000000000000000000000000000000 P e078a58a1b894aeeb8c58da9aa6aa920 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e078a58a1b894aeeb8c58da9aa6aa920" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 36391 } }
I20260810 18:49:10.504253 30256 raft_consensus.cc:388] T 00000000000000000000000000000000 P e078a58a1b894aeeb8c58da9aa6aa920 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:49:10.504320 30256 raft_consensus.cc:482] T 00000000000000000000000000000000 P e078a58a1b894aeeb8c58da9aa6aa920 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:49:10.504434 30256 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e078a58a1b894aeeb8c58da9aa6aa920 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:10.505390 30256 raft_consensus.cc:504] T 00000000000000000000000000000000 P e078a58a1b894aeeb8c58da9aa6aa920 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e078a58a1b894aeeb8c58da9aa6aa920" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 36391 } }
I20260810 18:49:10.505736 30256 leader_election.cc:302] T 00000000000000000000000000000000 P e078a58a1b894aeeb8c58da9aa6aa920 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e078a58a1b894aeeb8c58da9aa6aa920; no voters:
I20260810 18:49:10.506018 30256 leader_election.cc:288] T 00000000000000000000000000000000 P e078a58a1b894aeeb8c58da9aa6aa920 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:49:10.506150 30263 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e078a58a1b894aeeb8c58da9aa6aa920 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:49:10.506393 30263 raft_consensus.cc:686] T 00000000000000000000000000000000 P e078a58a1b894aeeb8c58da9aa6aa920 [term 1 LEADER]: Becoming Leader. State: Replica: e078a58a1b894aeeb8c58da9aa6aa920, State: Running, Role: LEADER
I20260810 18:49:10.506708 30263 consensus_queue.cc:237] T 00000000000000000000000000000000 P e078a58a1b894aeeb8c58da9aa6aa920 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e078a58a1b894aeeb8c58da9aa6aa920" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 36391 } }
I20260810 18:49:10.507293 30256 sys_catalog.cc:564] T 00000000000000000000000000000000 P e078a58a1b894aeeb8c58da9aa6aa920 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:49:10.508270 30268 sys_catalog.cc:455] T 00000000000000000000000000000000 P e078a58a1b894aeeb8c58da9aa6aa920 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e078a58a1b894aeeb8c58da9aa6aa920. Latest consensus state: current_term: 1 leader_uuid: "e078a58a1b894aeeb8c58da9aa6aa920" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e078a58a1b894aeeb8c58da9aa6aa920" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 36391 } } }
I20260810 18:49:10.508404 30268 sys_catalog.cc:458] T 00000000000000000000000000000000 P e078a58a1b894aeeb8c58da9aa6aa920 [sys.catalog]: This master's current role is: LEADER
I20260810 18:49:10.508891 30264 sys_catalog.cc:455] T 00000000000000000000000000000000 P e078a58a1b894aeeb8c58da9aa6aa920 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e078a58a1b894aeeb8c58da9aa6aa920" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e078a58a1b894aeeb8c58da9aa6aa920" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 36391 } } }
I20260810 18:49:10.509047 30264 sys_catalog.cc:458] T 00000000000000000000000000000000 P e078a58a1b894aeeb8c58da9aa6aa920 [sys.catalog]: This master's current role is: LEADER
W20260810 18:49:10.511977 30284 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e078a58a1b894aeeb8c58da9aa6aa920: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:49:10.512044 30284 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:49:10.512140 30275 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:49:10.512924 30275 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:49:10.518392 30275 catalog_manager.cc:1347] Generated new cluster ID: d992180d72a944c299db78c60432f88b
I20260810 18:49:10.518451 30275 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:49:10.540077 30275 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:49:10.540405 30275 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:49:10.548599 30275 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e078a58a1b894aeeb8c58da9aa6aa920: Generated new TSK 0
I20260810 18:49:10.549393 30275 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:49:10.561729 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.129:0
--local_ip_for_outbound_sockets=127.18.168.129
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:36391
--builtin_ntp_servers=127.18.168.148:41815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 18:49:10.673223 30287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:10.673461 30287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:10.673498 30287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:10.677444 30287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:10.677489 30287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:10.677587 30287 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.129
I20260810 18:49:10.682158 30287 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:41815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:36391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.129
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:49:10.683063 30287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:10.684310 30287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:10.693711 30293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:10.693730 30292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:10.693714 30295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:10.694048 30287 server_base.cc:1034] running on GCE node
I20260810 18:49:10.694394 30287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:10.694953 30287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:10.696130 30287 hybrid_clock.cc:648] HybridClock initialized: now 1786387750696080 us; error 41 us; skew 500 ppm
I20260810 18:49:10.698047 30287 webserver.cc:460] Webserver started at http://127.18.168.129:37573/ using document root <none> and password file <none>
I20260810 18:49:10.698604 30287 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:10.698685 30287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:10.698899 30287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:10.700838 30287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/data/instance:
uuid: "357bbdb5feba4b1cb0f10bb421371092"
format_stamp: "Formatted at 2026-08-10 18:49:10 on dist-test-slave-cbsf"
server_key: "949746e6c139cb46487c9e5b44b7e95e"
server_key_iv: "e9d5a7047660589bbc71f279154e1fac"
server_key_version: "encryptionkey@0"
I20260810 18:49:10.701412 30287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance:
uuid: "357bbdb5feba4b1cb0f10bb421371092"
format_stamp: "Formatted at 2026-08-10 18:49:10 on dist-test-slave-cbsf"
server_key: "949746e6c139cb46487c9e5b44b7e95e"
server_key_iv: "e9d5a7047660589bbc71f279154e1fac"
server_key_version: "encryptionkey@0"
I20260810 18:49:10.705045 30287 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260810 18:49:10.707547 30303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:10.708985 30287 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:49:10.709101 30287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/wal
uuid: "357bbdb5feba4b1cb0f10bb421371092"
format_stamp: "Formatted at 2026-08-10 18:49:10 on dist-test-slave-cbsf"
server_key: "949746e6c139cb46487c9e5b44b7e95e"
server_key_iv: "e9d5a7047660589bbc71f279154e1fac"
server_key_version: "encryptionkey@0"
I20260810 18:49:10.709226 30287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:10.713671 30287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:10.714264 30287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:10.714442 30287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:10.714964 30287 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:49:10.715898 30287 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:49:10.715965 30287 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:10.716028 30287 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:49:10.716074 30287 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:10.724653 30287 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.129:38491
I20260810 18:49:10.724675 30435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.129:38491 every 8 connection(s)
I20260810 18:49:10.725533 30287 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
I20260810 18:49:10.728442 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 30287
I20260810 18:49:10.728571 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance
I20260810 18:49:10.728914 19106 external_mini_cluster.cc:1428] Setting key bebd6ccceb13e16c6256b4716e9dc374
I20260810 18:49:10.731145 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.130:0
--local_ip_for_outbound_sockets=127.18.168.130
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:36391
--builtin_ntp_servers=127.18.168.148:41815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260810 18:49:10.731932 30259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:10.727304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:56031 (local address 127.18.168.190:36391)
0810 18:49:10.727466 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:10.727471 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:10.728450 (+ 979us) server_negotiation.cc:407] Connection header received
0810 18:49:10.730105 (+ 1655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:10.730107 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:10.730146 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:10.730227 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:10.731503 (+ 1276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:10.731507 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:10.731509 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:49:10.731522 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:10.731544 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:49:10.731552 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:10.731630 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:10.731798 (+ 168us) server_negotiation.cc:299] Negotiation successful
0810 18:49:10.731834 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 18:49:10.733070 30439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:10.727632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:36391 (local address 127.18.168.129:56031)
0810 18:49:10.728278 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:10.728333 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:10.729900 (+ 1567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:10.730442 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:10.730452 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:10.730554 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:10.731119 (+ 565us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:49:10.731123 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:10.731198 (+ 75us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:49:10.731299 (+ 101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:10.731639 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:10.731650 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:10.731660 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 18:49:10.731836 (+ 176us) client_negotiation.cc:240] Negotiation successful
0810 18:49:10.732077 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":104,"threads_started":1}
I20260810 18:49:10.734299 30437 heartbeater.cc:347] Connected to a master server at 127.18.168.190:36391
I20260810 18:49:10.734515 30437 heartbeater.cc:464] Registering TS with master...
I20260810 18:49:10.734892 30437 heartbeater.cc:511] Master 127.18.168.190:36391 requested a full tablet report, sending...
I20260810 18:49:10.736395 30215 ts_manager.cc:194] Registered new tserver with Master: 357bbdb5feba4b1cb0f10bb421371092 (127.18.168.129:38491)
W20260810 18:49:10.842130 30440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:10.842393 30440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:10.842463 30440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:10.846406 30440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:10.846472 30440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:10.846560 30440 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.130
I20260810 18:49:10.851106 30440 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:41815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:36391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.130
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:49:10.852118 30440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:10.853483 30440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:10.862795 30445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:10.862802 30446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:10.862802 30448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:10.863013 30440 server_base.cc:1034] running on GCE node
I20260810 18:49:10.863384 30440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:10.863997 30440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:10.865167 30440 hybrid_clock.cc:648] HybridClock initialized: now 1786387750865149 us; error 28 us; skew 500 ppm
I20260810 18:49:10.866880 30440 webserver.cc:460] Webserver started at http://127.18.168.130:37323/ using document root <none> and password file <none>
I20260810 18:49:10.867419 30440 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:10.867504 30440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:10.867726 30440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:10.869746 30440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/data/instance:
uuid: "4b94558de3ac4101a466e1a5b83bc794"
format_stamp: "Formatted at 2026-08-10 18:49:10 on dist-test-slave-cbsf"
server_key: "ed6cda58309d337db6fcd72377b50609"
server_key_iv: "c4c48d1b0ccddb34a730ad2b447054a4"
server_key_version: "encryptionkey@0"
I20260810 18:49:10.870344 30440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance:
uuid: "4b94558de3ac4101a466e1a5b83bc794"
format_stamp: "Formatted at 2026-08-10 18:49:10 on dist-test-slave-cbsf"
server_key: "ed6cda58309d337db6fcd72377b50609"
server_key_iv: "c4c48d1b0ccddb34a730ad2b447054a4"
server_key_version: "encryptionkey@0"
I20260810 18:49:10.873834 30440 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:49:10.876291 30454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:10.877485 30440 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:49:10.877628 30440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/wal
uuid: "4b94558de3ac4101a466e1a5b83bc794"
format_stamp: "Formatted at 2026-08-10 18:49:10 on dist-test-slave-cbsf"
server_key: "ed6cda58309d337db6fcd72377b50609"
server_key_iv: "c4c48d1b0ccddb34a730ad2b447054a4"
server_key_version: "encryptionkey@0"
I20260810 18:49:10.877733 30440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:10.882085 30440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:10.882635 30440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:10.882802 30440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:10.883332 30440 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:49:10.884183 30440 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:49:10.884258 30440 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:10.884327 30440 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:49:10.884373 30440 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:10.893605 30440 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.130:39239
I20260810 18:49:10.893627 30586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.130:39239 every 8 connection(s)
I20260810 18:49:10.894441 30440 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
I20260810 18:49:10.898599 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 30440
I20260810 18:49:10.898741 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance
I20260810 18:49:10.899080 19106 external_mini_cluster.cc:1428] Setting key c746f0721ab719579cd6fd095d9f2c23
I20260810 18:49:10.900807 30259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:10.896193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:46649 (local address 127.18.168.190:36391)
0810 18:49:10.896359 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:10.896364 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:10.897438 (+ 1074us) server_negotiation.cc:407] Connection header received
0810 18:49:10.898900 (+ 1462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:10.898906 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:10.898958 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:10.899062 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:10.900291 (+ 1229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:10.900294 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:10.900296 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:49:10.900306 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:10.900326 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:49:10.900335 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:10.900433 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:10.900592 (+ 159us) server_negotiation.cc:299] Negotiation successful
0810 18:49:10.900629 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 18:49:10.901319 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.131:0
--local_ip_for_outbound_sockets=127.18.168.131
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:36391
--builtin_ntp_servers=127.18.168.148:41815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260810 18:49:10.901686 30589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:10.896595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:36391 (local address 127.18.168.130:46649)
0810 18:49:10.897295 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:10.897343 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:10.898646 (+ 1303us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:10.899235 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:10.899244 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:10.899349 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:10.899999 (+ 650us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:49:10.900001 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:10.900062 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:49:10.900144 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:10.900423 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:10.900433 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:10.900439 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 18:49:10.900625 (+ 186us) client_negotiation.cc:240] Negotiation successful
0810 18:49:10.900879 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":125,"threads_started":1}
I20260810 18:49:10.903892 30587 heartbeater.cc:347] Connected to a master server at 127.18.168.190:36391
I20260810 18:49:10.904124 30587 heartbeater.cc:464] Registering TS with master...
I20260810 18:49:10.904520 30587 heartbeater.cc:511] Master 127.18.168.190:36391 requested a full tablet report, sending...
I20260810 18:49:10.905615 30215 ts_manager.cc:194] Registered new tserver with Master: 4b94558de3ac4101a466e1a5b83bc794 (127.18.168.130:39239)
W20260810 18:49:11.016880 30590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:11.017119 30590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:11.017174 30590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:11.021283 30590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:11.021337 30590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:11.021404 30590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.131
I20260810 18:49:11.026139 30590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:41815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:36391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.131
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:49:11.027187 30590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:11.028594 30590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:11.037953 30598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:11.037997 30597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:11.037957 30601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:11.038244 30590 server_base.cc:1034] running on GCE node
I20260810 18:49:11.038902 30590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:11.039489 30590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:11.040665 30590 hybrid_clock.cc:648] HybridClock initialized: now 1786387751040637 us; error 40 us; skew 500 ppm
I20260810 18:49:11.042464 30590 webserver.cc:460] Webserver started at http://127.18.168.131:37519/ using document root <none> and password file <none>
I20260810 18:49:11.043018 30590 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:11.043107 30590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:11.043335 30590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:11.045446 30590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/data/instance:
uuid: "9815c308bb3b4778b19351770161bc45"
format_stamp: "Formatted at 2026-08-10 18:49:11 on dist-test-slave-cbsf"
server_key: "9d53433431f3bde542f379f5fb62ea03"
server_key_iv: "eefe898489b60cd312d461ae57f20806"
server_key_version: "encryptionkey@0"
I20260810 18:49:11.046038 30590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance:
uuid: "9815c308bb3b4778b19351770161bc45"
format_stamp: "Formatted at 2026-08-10 18:49:11 on dist-test-slave-cbsf"
server_key: "9d53433431f3bde542f379f5fb62ea03"
server_key_iv: "eefe898489b60cd312d461ae57f20806"
server_key_version: "encryptionkey@0"
I20260810 18:49:11.049840 30590 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:49:11.052373 30608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:11.053567 30590 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:49:11.053701 30590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/wal
uuid: "9815c308bb3b4778b19351770161bc45"
format_stamp: "Formatted at 2026-08-10 18:49:11 on dist-test-slave-cbsf"
server_key: "9d53433431f3bde542f379f5fb62ea03"
server_key_iv: "eefe898489b60cd312d461ae57f20806"
server_key_version: "encryptionkey@0"
I20260810 18:49:11.053817 30590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:11.058378 30590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:11.058976 30590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:11.059166 30590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:11.059746 30590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:49:11.060595 30590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:49:11.060643 30590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:11.060683 30590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:49:11.060760 30590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:11.069306 30590 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.131:37779
I20260810 18:49:11.069372 30745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.131:37779 every 8 connection(s)
I20260810 18:49:11.070272 30590 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
I20260810 18:49:11.076371 30259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:11.072149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:59905 (local address 127.18.168.190:36391)
0810 18:49:11.072304 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:11.072309 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:11.073480 (+ 1171us) server_negotiation.cc:407] Connection header received
0810 18:49:11.074728 (+ 1248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:11.074730 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:11.074768 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:11.074843 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:11.075987 (+ 1144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:11.075990 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:11.075992 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:49:11.076003 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:11.076023 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:49:11.076031 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:11.076107 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:11.076224 (+ 117us) server_negotiation.cc:299] Negotiation successful
0810 18:49:11.076261 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 18:49:11.077351 30748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:11.072603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:36391 (local address 127.18.168.131:59905)
0810 18:49:11.073339 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:11.073379 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:11.074546 (+ 1167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:11.074999 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:11.075006 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:11.075099 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:11.075708 (+ 609us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:49:11.075710 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:11.075761 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:49:11.075853 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:11.076105 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:11.076111 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:11.076115 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 18:49:11.076245 (+ 130us) client_negotiation.cc:240] Negotiation successful
0810 18:49:11.076523 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":132,"threads_started":1}
I20260810 18:49:11.078462 30746 heartbeater.cc:347] Connected to a master server at 127.18.168.190:36391
I20260810 18:49:11.078516 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 30590
I20260810 18:49:11.078630 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance
I20260810 18:49:11.078723 30746 heartbeater.cc:464] Registering TS with master...
I20260810 18:49:11.078964 19106 external_mini_cluster.cc:1428] Setting key b779691e1bd997cf68d953dfd148c029
I20260810 18:49:11.079203 30746 heartbeater.cc:511] Master 127.18.168.190:36391 requested a full tablet report, sending...
I20260810 18:49:11.080356 30215 ts_manager.cc:194] Registered new tserver with Master: 9815c308bb3b4778b19351770161bc45 (127.18.168.131:37779)
I20260810 18:49:11.081177 19106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 18:49:11.083959 30259 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:11.083164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60808 (local address 127.18.168.190:36391)
0810 18:49:11.083289 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:11.083297 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:11.083482 (+ 185us) server_negotiation.cc:407] Connection header received
0810 18:49:11.083585 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:11.083587 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:11.083616 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:11.083683 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:11.083866 (+ 183us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60808: BlockingRecv error: recv got EOF from 127.0.0.1:60808 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 18:49:11.084265 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 30287
I20260810 18:49:11.100876 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 30440
I20260810 18:49:11.111785 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 30590
I20260810 18:49:11.121834 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 30184
2026-08-10T18:49:11Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (827 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:49:11 dist-test-slave-cbsf krb5kdc[30761](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:49:11 dist-test-slave-cbsf krb5kdc[30761](info): set up 2 sockets
Aug 10 18:49:11 dist-test-slave-cbsf krb5kdc[30761](info): commencing operation
krb5kdc: starting...
W20260810 18:49:12.232012 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:49:12 dist-test-slave-cbsf krb5kdc[30761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:49:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:49:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.168.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
I20260810 18:49:12.360281 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:41887
--webserver_interface=127.18.168.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:43407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:41887
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:49:12.469815 30775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:12.470108 30775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:12.470175 30775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:12.474113 30775 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:49:12.474207 30775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:12.474243 30775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:12.474299 30775 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:49:12.474339 30775 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:49:12.479708 30775 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:43407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:41887
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:41887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:49:12.480752 30775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:12.481779 30775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:12.489873 30782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:12.489928 30784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:12.489882 30780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:12.490188 30775 server_base.cc:1034] running on GCE node
I20260810 18:49:12.490804 30775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:12.491863 30775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:12.493048 30775 hybrid_clock.cc:648] HybridClock initialized: now 1786387752493017 us; error 44 us; skew 500 ppm
Aug 10 18:49:12 dist-test-slave-cbsf krb5kdc[30761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387752, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:12.495848 30775 init.cc:377] Logged in from keytab as kudu/127.18.168.190@KRBTEST.COM (short username kudu)
I20260810 18:49:12.496622 30775 webserver.cc:460] Webserver started at http://127.18.168.190:34541/ using document root <none> and password file <none>
I20260810 18:49:12.497192 30775 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:12.497283 30775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:12.497447 30775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:12.499389 30775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "45cf0694830e43fba2ddfffd6d51b2d1"
format_stamp: "Formatted at 2026-08-10 18:49:12 on dist-test-slave-cbsf"
server_key: "e100a91be2c992d92345f444ae725349"
server_key_iv: "b5d8ac2922e6d5c352cf95dd85c3e9e1"
server_key_version: "encryptionkey@0"
I20260810 18:49:12.499891 30775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "45cf0694830e43fba2ddfffd6d51b2d1"
format_stamp: "Formatted at 2026-08-10 18:49:12 on dist-test-slave-cbsf"
server_key: "e100a91be2c992d92345f444ae725349"
server_key_iv: "b5d8ac2922e6d5c352cf95dd85c3e9e1"
server_key_version: "encryptionkey@0"
I20260810 18:49:12.503518 30775 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260810 18:49:12.505872 30791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:12.507071 30775 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:49:12.507174 30775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "45cf0694830e43fba2ddfffd6d51b2d1"
format_stamp: "Formatted at 2026-08-10 18:49:12 on dist-test-slave-cbsf"
server_key: "e100a91be2c992d92345f444ae725349"
server_key_iv: "b5d8ac2922e6d5c352cf95dd85c3e9e1"
server_key_version: "encryptionkey@0"
I20260810 18:49:12.507256 30775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:12.521621 30775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:12.524209 30775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:12.524365 30775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:12.531275 30775 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:41887
I20260810 18:49:12.531561 30858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:41887 every 8 connection(s)
I20260810 18:49:12.532285 30775 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:49:12.535210 30859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:12.536423 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 30775
I20260810 18:49:12.536553 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:49:12.536902 19106 external_mini_cluster.cc:1428] Setting key cb2a8331c8e3b8f3096fde6e84587963
I20260810 18:49:12.541695 30859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45cf0694830e43fba2ddfffd6d51b2d1: Bootstrap starting.
I20260810 18:49:12.544397 30859 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45cf0694830e43fba2ddfffd6d51b2d1: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:12.545413 30859 log.cc:824] T 00000000000000000000000000000000 P 45cf0694830e43fba2ddfffd6d51b2d1: Log is configured to *not* fsync() on all Append() calls
Aug 10 18:49:12 dist-test-slave-cbsf krb5kdc[30761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:49:12.547457 30859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45cf0694830e43fba2ddfffd6d51b2d1: No bootstrap required, opened a new log
I20260810 18:49:12.551844 30862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:12.538299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33216 (local address 127.18.168.190:41887)
0810 18:49:12.538681 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:12.538706 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:12.538741 (+ 35us) server_negotiation.cc:407] Connection header received
0810 18:49:12.539381 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:12.539397 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:12.539698 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:12.539956 (+ 258us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:12.540880 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.541887 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.543048 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.543517 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.543629 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:12.546654 (+ 3025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:12.546659 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:12.546672 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:12.546698 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:12.548736 (+ 2038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:12.549145 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:12.549152 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:12.549159 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:12.549294 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:12.549578 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:12.549581 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:12.549583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:12.550188 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:12.550385 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:12.550611 (+ 226us) server_negotiation.cc:299] Negotiation successful
0810 18:49:12.550907 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":126,"threads_started":1}
I20260810 18:49:12.552139 30859 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45cf0694830e43fba2ddfffd6d51b2d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45cf0694830e43fba2ddfffd6d51b2d1" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41887 } }
I20260810 18:49:12.552372 30859 raft_consensus.cc:374] T 00000000000000000000000000000000 P 45cf0694830e43fba2ddfffd6d51b2d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:12.552438 30859 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45cf0694830e43fba2ddfffd6d51b2d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45cf0694830e43fba2ddfffd6d51b2d1, State: Initialized, Role: FOLLOWER
I20260810 18:49:12.552888 30859 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45cf0694830e43fba2ddfffd6d51b2d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45cf0694830e43fba2ddfffd6d51b2d1" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41887 } }
I20260810 18:49:12.553038 30859 raft_consensus.cc:388] T 00000000000000000000000000000000 P 45cf0694830e43fba2ddfffd6d51b2d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:49:12.553122 30859 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45cf0694830e43fba2ddfffd6d51b2d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:49:12.553256 30859 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45cf0694830e43fba2ddfffd6d51b2d1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:12.554124 30859 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45cf0694830e43fba2ddfffd6d51b2d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45cf0694830e43fba2ddfffd6d51b2d1" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41887 } }
I20260810 18:49:12.554467 30859 leader_election.cc:302] T 00000000000000000000000000000000 P 45cf0694830e43fba2ddfffd6d51b2d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 45cf0694830e43fba2ddfffd6d51b2d1; no voters:
I20260810 18:49:12.554811 30859 leader_election.cc:288] T 00000000000000000000000000000000 P 45cf0694830e43fba2ddfffd6d51b2d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:49:12.554925 30864 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 45cf0694830e43fba2ddfffd6d51b2d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:49:12.555178 30864 raft_consensus.cc:686] T 00000000000000000000000000000000 P 45cf0694830e43fba2ddfffd6d51b2d1 [term 1 LEADER]: Becoming Leader. State: Replica: 45cf0694830e43fba2ddfffd6d51b2d1, State: Running, Role: LEADER
I20260810 18:49:12.555557 30864 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45cf0694830e43fba2ddfffd6d51b2d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45cf0694830e43fba2ddfffd6d51b2d1" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41887 } }
I20260810 18:49:12.555985 30859 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45cf0694830e43fba2ddfffd6d51b2d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:49:12.557201 30866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45cf0694830e43fba2ddfffd6d51b2d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45cf0694830e43fba2ddfffd6d51b2d1. Latest consensus state: current_term: 1 leader_uuid: "45cf0694830e43fba2ddfffd6d51b2d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45cf0694830e43fba2ddfffd6d51b2d1" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41887 } } }
I20260810 18:49:12.557663 30866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45cf0694830e43fba2ddfffd6d51b2d1 [sys.catalog]: This master's current role is: LEADER
I20260810 18:49:12.557526 30865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45cf0694830e43fba2ddfffd6d51b2d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "45cf0694830e43fba2ddfffd6d51b2d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45cf0694830e43fba2ddfffd6d51b2d1" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41887 } } }
I20260810 18:49:12.557852 30865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45cf0694830e43fba2ddfffd6d51b2d1 [sys.catalog]: This master's current role is: LEADER
I20260810 18:49:12.558777 30877 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:49:12.561393 30877 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:49:12.566551 30877 catalog_manager.cc:1347] Generated new cluster ID: db2db6cce73b4c0ca98d94cf0cdb124d
I20260810 18:49:12.566622 30877 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:49:12.575660 30877 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:49:12.576838 30877 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:49:12.589619 30877 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 45cf0694830e43fba2ddfffd6d51b2d1: Generated new TSK 0
I20260810 18:49:12.590236 30877 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.168.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab.
Entry for principal kudu/127.18.168.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab.
I20260810 18:49:12.620332 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.129:0
--local_ip_for_outbound_sockets=127.18.168.129
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:41887
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:43407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:49:12.747630 30887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:12.747892 30887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:12.747963 30887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:12.751821 30887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:12.751899 30887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:12.751987 30887 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.129
I20260810 18:49:12.757398 30887 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:43407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:41887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.129
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:49:12.758458 30887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:12.759411 30887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:12.768883 30894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:12.768859 30897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:12.768826 30895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:12.768849 30887 server_base.cc:1034] running on GCE node
I20260810 18:49:12.769394 30887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:12.770016 30887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:12.771204 30887 hybrid_clock.cc:648] HybridClock initialized: now 1786387752771192 us; error 24 us; skew 500 ppm
Aug 10 18:49:12 dist-test-slave-cbsf krb5kdc[30761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387752, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:12.773896 30887 init.cc:377] Logged in from keytab as kudu/127.18.168.129@KRBTEST.COM (short username kudu)
I20260810 18:49:12.774745 30887 webserver.cc:460] Webserver started at http://127.18.168.129:35589/ using document root <none> and password file <none>
I20260810 18:49:12.775306 30887 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:12.775400 30887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:12.775663 30887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:12.777832 30887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/data/instance:
uuid: "d5b37cd0e4504582981d3f91cb87c2d0"
format_stamp: "Formatted at 2026-08-10 18:49:12 on dist-test-slave-cbsf"
server_key: "7df95e0d22f70717f5df02c5c313557e"
server_key_iv: "a667b778052e4241cda7b13ba2c36936"
server_key_version: "encryptionkey@0"
I20260810 18:49:12.778394 30887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance:
uuid: "d5b37cd0e4504582981d3f91cb87c2d0"
format_stamp: "Formatted at 2026-08-10 18:49:12 on dist-test-slave-cbsf"
server_key: "7df95e0d22f70717f5df02c5c313557e"
server_key_iv: "a667b778052e4241cda7b13ba2c36936"
server_key_version: "encryptionkey@0"
I20260810 18:49:12.781975 30887 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:49:12.784356 30905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:12.785564 30887 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:49:12.785727 30887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/wal
uuid: "d5b37cd0e4504582981d3f91cb87c2d0"
format_stamp: "Formatted at 2026-08-10 18:49:12 on dist-test-slave-cbsf"
server_key: "7df95e0d22f70717f5df02c5c313557e"
server_key_iv: "a667b778052e4241cda7b13ba2c36936"
server_key_version: "encryptionkey@0"
I20260810 18:49:12.785835 30887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:12.806388 30887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:12.809264 30887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:12.809461 30887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:12.810168 30887 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:49:12.811070 30887 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:49:12.811167 30887 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:12.811244 30887 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:49:12.811277 30887 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:12.820160 30887 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.129:34175
I20260810 18:49:12.820185 31041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.129:34175 every 8 connection(s)
I20260810 18:49:12.821115 30887 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
I20260810 18:49:12.827152 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 30887
I20260810 18:49:12.827247 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance
I20260810 18:49:12.827536 19106 external_mini_cluster.cc:1428] Setting key 57d3742708dd2d3ddff528efe9397f54
Aug 10 18:49:12 dist-test-slave-cbsf krb5kdc[30761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387752, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:49:12.834636 30862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:12.822900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:47335 (local address 127.18.168.190:41887)
0810 18:49:12.823062 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:12.823066 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:12.824090 (+ 1024us) server_negotiation.cc:407] Connection header received
0810 18:49:12.825054 (+ 964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:12.825057 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:12.825121 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:12.825225 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:12.826360 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.826892 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.828207 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.828452 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.828568 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:12.831058 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:12.831061 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:12.831064 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:12.831086 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:12.832588 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:12.833115 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:12.833119 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:12.833121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:12.833201 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:12.833545 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:12.833548 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:12.833550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:12.833729 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:12.833836 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:12.834320 (+ 484us) server_negotiation.cc:299] Negotiation successful
0810 18:49:12.834464 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 18:49:12.835429 31044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:12.823307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41887 (local address 127.18.168.129:47335)
0810 18:49:12.823941 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:12.823981 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:12.824842 (+ 861us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:12.825400 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:12.825406 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:12.825755 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:12.826211 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:12.826224 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.827036 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.827040 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:12.828044 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:12.828054 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.828609 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.828613 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:12.828840 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:12.829435 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:12.829451 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:12.830880 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:12.832739 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:12.832744 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:12.832758 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:12.833009 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:12.833295 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:12.833299 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:12.833300 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:12.833440 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:12.833836 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:12.833843 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:12.834083 (+ 240us) client_negotiation.cc:769] Sending connection context
0810 18:49:12.834256 (+ 173us) client_negotiation.cc:240] Negotiation successful
0810 18:49:12.834607 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":100,"threads_started":1}
I20260810 18:49:12.836629 31042 heartbeater.cc:347] Connected to a master server at 127.18.168.190:41887
I20260810 18:49:12.836915 31042 heartbeater.cc:464] Registering TS with master...
I20260810 18:49:12.837559 31042 heartbeater.cc:511] Master 127.18.168.190:41887 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.168.130@KRBTEST.COM; defaulting to no policy
I20260810 18:49:12.839169 30810 ts_manager.cc:194] Registered new tserver with Master: d5b37cd0e4504582981d3f91cb87c2d0 (127.18.168.129:34175)
I20260810 18:49:12.840433 30810 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.129@KRBTEST.COM'} at 127.18.168.129:47335
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab.
Entry for principal kudu/127.18.168.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab.
I20260810 18:49:12.855651 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.130:0
--local_ip_for_outbound_sockets=127.18.168.130
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:41887
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:43407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:49:12.965296 31047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:12.965575 31047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:12.965618 31047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:12.969514 31047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:12.969560 31047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:12.969641 31047 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.130
I20260810 18:49:12.974887 31047 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:43407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:41887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.130
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:49:12.975847 31047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:12.976856 31047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:12.986384 31057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:12.986444 31054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:12.986378 31055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:12.986826 31047 server_base.cc:1034] running on GCE node
I20260810 18:49:12.987249 31047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:12.987960 31047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:12.989166 31047 hybrid_clock.cc:648] HybridClock initialized: now 1786387752989132 us; error 43 us; skew 500 ppm
Aug 10 18:49:12 dist-test-slave-cbsf krb5kdc[30761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387752, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:12.992067 31047 init.cc:377] Logged in from keytab as kudu/127.18.168.130@KRBTEST.COM (short username kudu)
I20260810 18:49:12.993024 31047 webserver.cc:460] Webserver started at http://127.18.168.130:36881/ using document root <none> and password file <none>
I20260810 18:49:12.993588 31047 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:12.993686 31047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:12.993914 31047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:12.996042 31047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/data/instance:
uuid: "aae9b99504854b59bd4ba8cc5ef5d891"
format_stamp: "Formatted at 2026-08-10 18:49:12 on dist-test-slave-cbsf"
server_key: "7f35a4c8a06819c5cfc9d1cbe77ddf76"
server_key_iv: "92c7a54a73b233c06e258baaf89eca46"
server_key_version: "encryptionkey@0"
I20260810 18:49:12.996625 31047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance:
uuid: "aae9b99504854b59bd4ba8cc5ef5d891"
format_stamp: "Formatted at 2026-08-10 18:49:12 on dist-test-slave-cbsf"
server_key: "7f35a4c8a06819c5cfc9d1cbe77ddf76"
server_key_iv: "92c7a54a73b233c06e258baaf89eca46"
server_key_version: "encryptionkey@0"
I20260810 18:49:13.002305 31047 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 18:49:13.007241 31065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:13.009286 31047 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260810 18:49:13.009416 31047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/wal
uuid: "aae9b99504854b59bd4ba8cc5ef5d891"
format_stamp: "Formatted at 2026-08-10 18:49:12 on dist-test-slave-cbsf"
server_key: "7f35a4c8a06819c5cfc9d1cbe77ddf76"
server_key_iv: "92c7a54a73b233c06e258baaf89eca46"
server_key_version: "encryptionkey@0"
I20260810 18:49:13.009538 31047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:13.039749 31047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:13.043437 31047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:13.043666 31047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:13.044317 31047 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:49:13.045310 31047 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:49:13.045361 31047 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:13.045434 31047 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:49:13.045475 31047 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:13.055294 31047 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.130:46881
I20260810 18:49:13.055331 31200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.130:46881 every 8 connection(s)
I20260810 18:49:13.056355 31047 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
I20260810 18:49:13.062362 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 31047
I20260810 18:49:13.062458 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance
I20260810 18:49:13.062743 19106 external_mini_cluster.cc:1428] Setting key 551f8ee28a4233efe5e3fbe1cd57f55c
Aug 10 18:49:13 dist-test-slave-cbsf krb5kdc[30761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387752, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.168.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.131@KRBTEST.COM" created.
I20260810 18:49:13.078603 30862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:13.058382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:33707 (local address 127.18.168.190:41887)
0810 18:49:13.058595 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:13.058600 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:13.059627 (+ 1027us) server_negotiation.cc:407] Connection header received
0810 18:49:13.061229 (+ 1602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:13.061233 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:13.061307 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:13.061394 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:13.062655 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.063446 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.064878 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.065130 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.065249 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:13.068211 (+ 2962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:13.068222 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:13.068224 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:13.068250 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:13.076075 (+ 7825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:13.076781 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:13.076784 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:13.076785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:13.076863 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:13.077317 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:13.077319 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:13.077321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:13.077571 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:13.077694 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:13.078226 (+ 532us) server_negotiation.cc:299] Negotiation successful
0810 18:49:13.078395 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260810 18:49:13.079332 31203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:13.058953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41887 (local address 127.18.168.130:33707)
0810 18:49:13.059426 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:13.059512 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:13.061006 (+ 1494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:13.061576 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:13.061583 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:13.061952 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:13.062487 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:13.062518 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.063582 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.063586 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:13.064676 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:13.064738 (+ 62us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.065290 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.065295 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:13.065487 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:13.066123 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:13.066140 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:13.068033 (+ 1893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:13.076201 (+ 8168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:13.076210 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:13.076230 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:13.076643 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:13.076957 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:13.076962 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:13.076967 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:13.077190 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:13.077700 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:13.077706 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:13.077964 (+ 258us) client_negotiation.cc:769] Sending connection context
0810 18:49:13.078126 (+ 162us) client_negotiation.cc:240] Negotiation successful
0810 18:49:13.078410 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":111,"threads_started":1}
I20260810 18:49:13.080600 31201 heartbeater.cc:347] Connected to a master server at 127.18.168.190:41887
I20260810 18:49:13.080897 31201 heartbeater.cc:464] Registering TS with master...
I20260810 18:49:13.081380 31201 heartbeater.cc:511] Master 127.18.168.190:41887 requested a full tablet report, sending...
I20260810 18:49:13.082490 30810 ts_manager.cc:194] Registered new tserver with Master: aae9b99504854b59bd4ba8cc5ef5d891 (127.18.168.130:46881)
I20260810 18:49:13.083094 30810 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.130@KRBTEST.COM'} at 127.18.168.130:33707
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab.
Entry for principal kudu/127.18.168.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab.
I20260810 18:49:13.092414 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.131:0
--local_ip_for_outbound_sockets=127.18.168.131
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:41887
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:43407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:49:13.201658 31207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:13.201917 31207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:13.201997 31207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:13.205915 31207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:13.205989 31207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:13.206087 31207 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.131
I20260810 18:49:13.211213 31207 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:43407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:41887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.131
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:49:13.212193 31207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:13.213287 31207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:13.222751 31215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:13.222751 31214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:13.222982 31207 server_base.cc:1034] running on GCE node
W20260810 18:49:13.222785 31218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:13.223441 31207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:13.224115 31207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:13.225297 31207 hybrid_clock.cc:648] HybridClock initialized: now 1786387753225276 us; error 29 us; skew 500 ppm
Aug 10 18:49:13 dist-test-slave-cbsf krb5kdc[30761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387753, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:13.228087 31207 init.cc:377] Logged in from keytab as kudu/127.18.168.131@KRBTEST.COM (short username kudu)
I20260810 18:49:13.228986 31207 webserver.cc:460] Webserver started at http://127.18.168.131:40205/ using document root <none> and password file <none>
I20260810 18:49:13.229517 31207 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:13.229573 31207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:13.229737 31207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:13.231626 31207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/data/instance:
uuid: "f8f7e4fbfccc4de7a7bc51d60f6da4a9"
format_stamp: "Formatted at 2026-08-10 18:49:13 on dist-test-slave-cbsf"
server_key: "287f6f0d643f9dd46c2d3ec6d68e16fd"
server_key_iv: "f76de8fd2aef35fd2ddd5950c46f167e"
server_key_version: "encryptionkey@0"
I20260810 18:49:13.232108 31207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance:
uuid: "f8f7e4fbfccc4de7a7bc51d60f6da4a9"
format_stamp: "Formatted at 2026-08-10 18:49:13 on dist-test-slave-cbsf"
server_key: "287f6f0d643f9dd46c2d3ec6d68e16fd"
server_key_iv: "f76de8fd2aef35fd2ddd5950c46f167e"
server_key_version: "encryptionkey@0"
I20260810 18:49:13.235607 31207 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260810 18:49:13.238104 31225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:13.239634 31207 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 18:49:13.239804 31207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/wal
uuid: "f8f7e4fbfccc4de7a7bc51d60f6da4a9"
format_stamp: "Formatted at 2026-08-10 18:49:13 on dist-test-slave-cbsf"
server_key: "287f6f0d643f9dd46c2d3ec6d68e16fd"
server_key_iv: "f76de8fd2aef35fd2ddd5950c46f167e"
server_key_version: "encryptionkey@0"
I20260810 18:49:13.239928 31207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:13.254050 31207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:13.256946 31207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:13.257146 31207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:13.257722 31207 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:49:13.258540 31207 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:49:13.258607 31207 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:13.258677 31207 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:49:13.258726 31207 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:13.267009 31207 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.131:45065
I20260810 18:49:13.267074 31366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.131:45065 every 8 connection(s)
I20260810 18:49:13.267867 31207 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
I20260810 18:49:13.268997 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 31207
I20260810 18:49:13.269120 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance
I20260810 18:49:13.269443 19106 external_mini_cluster.cc:1428] Setting key 025545274e15b7fe460714ecfca43cd7
Aug 10 18:49:13 dist-test-slave-cbsf krb5kdc[30761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387753, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:49:13.281801 30862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:13.269682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:36709 (local address 127.18.168.190:41887)
0810 18:49:13.269839 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:13.269844 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:13.270768 (+ 924us) server_negotiation.cc:407] Connection header received
0810 18:49:13.272459 (+ 1691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:13.272461 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:13.272506 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:13.272571 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:13.273746 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.274292 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.275598 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.275812 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.275890 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:13.278375 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:13.278391 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:13.278393 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:13.278416 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:13.279885 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:13.280344 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:13.280347 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:13.280348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:13.280422 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:13.280786 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:13.280788 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:13.280789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:13.281044 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:13.281123 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:13.281570 (+ 447us) server_negotiation.cc:299] Negotiation successful
0810 18:49:13.281688 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 18:49:13.282609 31369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:13.270044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41887 (local address 127.18.168.131:36709)
0810 18:49:13.270618 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:13.270685 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:13.272210 (+ 1525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:13.272764 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:13.272775 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:13.273113 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:13.273594 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:13.273607 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.274400 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.274403 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:13.275484 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:13.275494 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.275938 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.275943 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:13.276084 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:13.276751 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:13.276776 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:13.278208 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:13.280005 (+ 1797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:13.280009 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:13.280020 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:13.280257 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:13.280506 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:13.280508 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:13.280510 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:13.280647 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:13.281139 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:13.281144 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:13.281347 (+ 203us) client_negotiation.cc:769] Sending connection context
0810 18:49:13.281501 (+ 154us) client_negotiation.cc:240] Negotiation successful
0810 18:49:13.281758 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":140,"threads_started":1}
I20260810 18:49:13.283892 31367 heartbeater.cc:347] Connected to a master server at 127.18.168.190:41887
I20260810 18:49:13.284118 31367 heartbeater.cc:464] Registering TS with master...
I20260810 18:49:13.284597 31367 heartbeater.cc:511] Master 127.18.168.190:41887 requested a full tablet report, sending...
I20260810 18:49:13.285611 30810 ts_manager.cc:194] Registered new tserver with Master: f8f7e4fbfccc4de7a7bc51d60f6da4a9 (127.18.168.131:45065)
I20260810 18:49:13.286208 30810 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.131@KRBTEST.COM'} at 127.18.168.131:36709
I20260810 18:49:13.295374 19106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:49:13.304157 30862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:13.297449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33226 (local address 127.18.168.190:41887)
0810 18:49:13.297591 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:13.297598 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:13.297786 (+ 188us) server_negotiation.cc:407] Connection header received
0810 18:49:13.297869 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:13.297871 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:13.297907 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:13.297972 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:13.298594 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.299102 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.299945 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.300160 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.300240 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:13.301522 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:13.301526 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:13.301529 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:13.301552 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:13.302739 (+ 1187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:13.303126 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:13.303128 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:13.303130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:13.303199 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:13.303432 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:13.303434 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:13.303435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:13.303634 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:13.303737 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:13.303950 (+ 213us) server_negotiation.cc:299] Negotiation successful
0810 18:49:13.304048 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 18:49:13.306802 30810 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33226:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:49:13.310051 30810 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:49:13.324592 31381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:13.318264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:46881 (local address 127.0.0.1:47874)
0810 18:49:13.319298 (+ 1034us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:13.319352 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:13.319481 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:13.320002 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:13.320009 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:13.320033 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:49:13.320169 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:13.320180 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.321604 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.321607 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:13.323132 (+ 1525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:13.323137 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.323988 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.323992 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:13.324110 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:13.324148 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 18:49:13.324285 (+ 137us) client_negotiation.cc:240] Negotiation successful
0810 18:49:13.324451 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":913,"mutex_wait_us":214,"thread_start_us":83,"threads_started":1}
I20260810 18:49:13.324599 31383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:13.318880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47874 (local address 127.18.168.130:46881)
0810 18:49:13.319511 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:13.319524 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:13.319549 (+ 25us) server_negotiation.cc:407] Connection header received
0810 18:49:13.319632 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:13.319640 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:13.319843 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:13.319969 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:49:13.320306 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.321475 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.323271 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.323840 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.324001 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:13.324138 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:13.324315 (+ 177us) server_negotiation.cc:299] Negotiation successful
0810 18:49:13.324429 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":114,"threads_started":1}
I20260810 18:49:13.325008 31380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:13.318935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:34175 (local address 127.0.0.1:40578)
0810 18:49:13.319706 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:13.319719 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:13.319798 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:13.320654 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:13.320657 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:13.320676 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:49:13.320792 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:13.320798 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.322080 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.322084 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:13.322990 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:13.323000 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.324302 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.324306 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:13.324580 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:13.324640 (+ 60us) client_negotiation.cc:769] Sending connection context
0810 18:49:13.324774 (+ 134us) client_negotiation.cc:240] Negotiation successful
0810 18:49:13.324845 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":713,"mutex_wait_us":116,"thread_start_us":117,"threads_started":1}
I20260810 18:49:13.325049 31382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:13.318324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59358 (local address 127.18.168.131:45065)
0810 18:49:13.318620 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:13.318634 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:13.319445 (+ 811us) server_negotiation.cc:407] Connection header received
0810 18:49:13.319670 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:13.319675 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:13.319885 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:13.320045 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:49:13.320461 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.321355 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.323258 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.324068 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.324169 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:13.324267 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:13.324803 (+ 536us) server_negotiation.cc:299] Negotiation successful
0810 18:49:13.324896 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":119,"threads_started":1}
I20260810 18:49:13.325354 31384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:13.318202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.131:45065 (local address 127.0.0.1:59358)
0810 18:49:13.319298 (+ 1096us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:13.319352 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:13.319503 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:13.320227 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:13.320229 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:13.320251 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:49:13.320346 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:13.320350 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.321480 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.321487 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:13.323125 (+ 1638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:13.323137 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.324317 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.324320 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:13.324584 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:13.324641 (+ 57us) client_negotiation.cc:769] Sending connection context
0810 18:49:13.325166 (+ 525us) client_negotiation.cc:240] Negotiation successful
0810 18:49:13.325240 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":917,"mutex_wait_us":249,"thread_start_us":76,"threads_started":1}
I20260810 18:49:13.325413 31385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:13.319122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40578 (local address 127.18.168.129:34175)
0810 18:49:13.320201 (+ 1079us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:13.320213 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:13.320229 (+ 16us) server_negotiation.cc:407] Connection header received
0810 18:49:13.320329 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:13.320334 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:13.320508 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:13.320659 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:49:13.320889 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.321927 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.323111 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.323971 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.324097 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:13.324233 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:13.325182 (+ 949us) server_negotiation.cc:299] Negotiation successful
0810 18:49:13.325277 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":992,"thread_start_us":70,"threads_started":1}
I20260810 18:49:13.327970 31122 tablet_service.cc:1467] Processing CreateTablet for tablet 2f6c7a351b0f4757b24a5e98fa0d4b09 (DEFAULT_TABLE table=test-table [id=133d1bae13fa487db616997736e209b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:49:13.328094 30963 tablet_service.cc:1467] Processing CreateTablet for tablet 2f6c7a351b0f4757b24a5e98fa0d4b09 (DEFAULT_TABLE table=test-table [id=133d1bae13fa487db616997736e209b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:49:13.328297 31278 tablet_service.cc:1467] Processing CreateTablet for tablet 2f6c7a351b0f4757b24a5e98fa0d4b09 (DEFAULT_TABLE table=test-table [id=133d1bae13fa487db616997736e209b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:49:13.329196 31122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f6c7a351b0f4757b24a5e98fa0d4b09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:13.329237 30963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f6c7a351b0f4757b24a5e98fa0d4b09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:13.329243 31278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f6c7a351b0f4757b24a5e98fa0d4b09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:13.335326 31386 tablet_bootstrap.cc:492] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P aae9b99504854b59bd4ba8cc5ef5d891: Bootstrap starting.
I20260810 18:49:13.336587 31387 tablet_bootstrap.cc:492] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P d5b37cd0e4504582981d3f91cb87c2d0: Bootstrap starting.
I20260810 18:49:13.337101 31388 tablet_bootstrap.cc:492] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P f8f7e4fbfccc4de7a7bc51d60f6da4a9: Bootstrap starting.
I20260810 18:49:13.338105 31386 tablet_bootstrap.cc:654] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P aae9b99504854b59bd4ba8cc5ef5d891: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:13.338660 31387 tablet_bootstrap.cc:654] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P d5b37cd0e4504582981d3f91cb87c2d0: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:13.338892 31386 log.cc:824] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P aae9b99504854b59bd4ba8cc5ef5d891: Log is configured to *not* fsync() on all Append() calls
I20260810 18:49:13.339080 31388 tablet_bootstrap.cc:654] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P f8f7e4fbfccc4de7a7bc51d60f6da4a9: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:13.339486 31387 log.cc:824] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P d5b37cd0e4504582981d3f91cb87c2d0: Log is configured to *not* fsync() on all Append() calls
I20260810 18:49:13.340258 31388 log.cc:824] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P f8f7e4fbfccc4de7a7bc51d60f6da4a9: Log is configured to *not* fsync() on all Append() calls
I20260810 18:49:13.341114 31386 tablet_bootstrap.cc:492] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P aae9b99504854b59bd4ba8cc5ef5d891: No bootstrap required, opened a new log
I20260810 18:49:13.341297 31387 tablet_bootstrap.cc:492] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P d5b37cd0e4504582981d3f91cb87c2d0: No bootstrap required, opened a new log
I20260810 18:49:13.341308 31386 ts_tablet_manager.cc:1397] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P aae9b99504854b59bd4ba8cc5ef5d891: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260810 18:49:13.341488 31387 ts_tablet_manager.cc:1397] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P d5b37cd0e4504582981d3f91cb87c2d0: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260810 18:49:13.342036 31388 tablet_bootstrap.cc:492] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P f8f7e4fbfccc4de7a7bc51d60f6da4a9: No bootstrap required, opened a new log
I20260810 18:49:13.342264 31388 ts_tablet_manager.cc:1397] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P f8f7e4fbfccc4de7a7bc51d60f6da4a9: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 18:49:13.345350 31388 raft_consensus.cc:348] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P f8f7e4fbfccc4de7a7bc51d60f6da4a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8f7e4fbfccc4de7a7bc51d60f6da4a9" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45065 } } peers { permanent_uuid: "aae9b99504854b59bd4ba8cc5ef5d891" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 46881 } } peers { permanent_uuid: "d5b37cd0e4504582981d3f91cb87c2d0" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 34175 } }
I20260810 18:49:13.345574 31388 raft_consensus.cc:374] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P f8f7e4fbfccc4de7a7bc51d60f6da4a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:13.345392 31386 raft_consensus.cc:348] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P aae9b99504854b59bd4ba8cc5ef5d891 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8f7e4fbfccc4de7a7bc51d60f6da4a9" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45065 } } peers { permanent_uuid: "aae9b99504854b59bd4ba8cc5ef5d891" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 46881 } } peers { permanent_uuid: "d5b37cd0e4504582981d3f91cb87c2d0" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 34175 } }
I20260810 18:49:13.345633 31388 raft_consensus.cc:729] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P f8f7e4fbfccc4de7a7bc51d60f6da4a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8f7e4fbfccc4de7a7bc51d60f6da4a9, State: Initialized, Role: FOLLOWER
I20260810 18:49:13.345644 31386 raft_consensus.cc:374] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P aae9b99504854b59bd4ba8cc5ef5d891 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:13.345706 31386 raft_consensus.cc:729] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P aae9b99504854b59bd4ba8cc5ef5d891 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aae9b99504854b59bd4ba8cc5ef5d891, State: Initialized, Role: FOLLOWER
I20260810 18:49:13.345768 31387 raft_consensus.cc:348] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P d5b37cd0e4504582981d3f91cb87c2d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8f7e4fbfccc4de7a7bc51d60f6da4a9" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45065 } } peers { permanent_uuid: "aae9b99504854b59bd4ba8cc5ef5d891" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 46881 } } peers { permanent_uuid: "d5b37cd0e4504582981d3f91cb87c2d0" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 34175 } }
I20260810 18:49:13.346091 31388 consensus_queue.cc:260] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P f8f7e4fbfccc4de7a7bc51d60f6da4a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8f7e4fbfccc4de7a7bc51d60f6da4a9" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45065 } } peers { permanent_uuid: "aae9b99504854b59bd4ba8cc5ef5d891" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 46881 } } peers { permanent_uuid: "d5b37cd0e4504582981d3f91cb87c2d0" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 34175 } }
I20260810 18:49:13.346102 31386 consensus_queue.cc:260] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P aae9b99504854b59bd4ba8cc5ef5d891 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8f7e4fbfccc4de7a7bc51d60f6da4a9" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45065 } } peers { permanent_uuid: "aae9b99504854b59bd4ba8cc5ef5d891" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 46881 } } peers { permanent_uuid: "d5b37cd0e4504582981d3f91cb87c2d0" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 34175 } }
I20260810 18:49:13.346696 31367 heartbeater.cc:503] Master 127.18.168.190:41887 was elected leader, sending a full tablet report...
I20260810 18:49:13.345978 31387 raft_consensus.cc:374] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P d5b37cd0e4504582981d3f91cb87c2d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:13.346786 31387 raft_consensus.cc:729] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P d5b37cd0e4504582981d3f91cb87c2d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5b37cd0e4504582981d3f91cb87c2d0, State: Initialized, Role: FOLLOWER
I20260810 18:49:13.347128 31386 ts_tablet_manager.cc:1428] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P aae9b99504854b59bd4ba8cc5ef5d891: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 18:49:13.347258 31387 consensus_queue.cc:260] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P d5b37cd0e4504582981d3f91cb87c2d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8f7e4fbfccc4de7a7bc51d60f6da4a9" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45065 } } peers { permanent_uuid: "aae9b99504854b59bd4ba8cc5ef5d891" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 46881 } } peers { permanent_uuid: "d5b37cd0e4504582981d3f91cb87c2d0" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 34175 } }
I20260810 18:49:13.347484 31201 heartbeater.cc:503] Master 127.18.168.190:41887 was elected leader, sending a full tablet report...
I20260810 18:49:13.347929 31388 ts_tablet_manager.cc:1428] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P f8f7e4fbfccc4de7a7bc51d60f6da4a9: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 18:49:13.348088 31387 ts_tablet_manager.cc:1428] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P d5b37cd0e4504582981d3f91cb87c2d0: Time spent starting tablet: real 0.007s user 0.000s sys 0.006s
I20260810 18:49:13.348395 31042 heartbeater.cc:503] Master 127.18.168.190:41887 was elected leader, sending a full tablet report...
I20260810 18:49:13.362267 31393 raft_consensus.cc:482] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P f8f7e4fbfccc4de7a7bc51d60f6da4a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:49:13.362465 31393 raft_consensus.cc:504] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P f8f7e4fbfccc4de7a7bc51d60f6da4a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8f7e4fbfccc4de7a7bc51d60f6da4a9" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45065 } } peers { permanent_uuid: "aae9b99504854b59bd4ba8cc5ef5d891" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 46881 } } peers { permanent_uuid: "d5b37cd0e4504582981d3f91cb87c2d0" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 34175 } }
I20260810 18:49:13.363531 31393 leader_election.cc:288] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P f8f7e4fbfccc4de7a7bc51d60f6da4a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aae9b99504854b59bd4ba8cc5ef5d891 (127.18.168.130:46881), d5b37cd0e4504582981d3f91cb87c2d0 (127.18.168.129:34175)
I20260810 18:49:13.367352 31383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:13.363857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:40065 (local address 127.18.168.130:46881)
0810 18:49:13.363983 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:13.363988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:13.364002 (+ 14us) server_negotiation.cc:407] Connection header received
0810 18:49:13.364054 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:13.364057 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:13.364110 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:13.364218 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:49:13.364545 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.365257 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.366399 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.366810 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.366939 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:13.366991 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:13.367172 (+ 181us) server_negotiation.cc:299] Negotiation successful
0810 18:49:13.367242 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 18:49:13.367352 31369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:13.363720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:46881 (local address 127.18.168.131:40065)
0810 18:49:13.363837 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:13.363854 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:13.363941 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:13.364283 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:13.364287 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:13.364311 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:49:13.364432 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:13.364441 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.365371 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.365377 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:13.366274 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:13.366283 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.366931 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.366935 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:13.367040 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:13.367067 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 18:49:13.367155 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 18:49:13.367225 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260810 18:49:13.367789 31396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:13.363992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:34175 (local address 127.18.168.131:46033)
0810 18:49:13.364337 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:13.364350 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:13.364471 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:13.364809 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:13.364812 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:13.364832 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:49:13.364949 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:13.364954 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.365734 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.365737 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:13.366658 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:13.366669 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.367310 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.367314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:13.367417 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:13.367451 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 18:49:13.367568 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 18:49:13.367630 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":125,"threads_started":1}
I20260810 18:49:13.367853 31385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:13.364103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:46033 (local address 127.18.168.129:34175)
0810 18:49:13.364240 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:13.364247 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:13.364457 (+ 210us) server_negotiation.cc:407] Connection header received
0810 18:49:13.364628 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:13.364631 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:13.364677 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:13.364772 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:49:13.365067 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.365563 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.366796 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.367187 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.367314 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:13.367362 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:13.367626 (+ 264us) server_negotiation.cc:299] Negotiation successful
0810 18:49:13.367724 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 18:49:13.368024 31151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f6c7a351b0f4757b24a5e98fa0d4b09" candidate_uuid: "f8f7e4fbfccc4de7a7bc51d60f6da4a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aae9b99504854b59bd4ba8cc5ef5d891" is_pre_election: true
I20260810 18:49:13.368347 31151 raft_consensus.cc:2454] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P aae9b99504854b59bd4ba8cc5ef5d891 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8f7e4fbfccc4de7a7bc51d60f6da4a9 in term 0.
I20260810 18:49:13.368402 30987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f6c7a351b0f4757b24a5e98fa0d4b09" candidate_uuid: "f8f7e4fbfccc4de7a7bc51d60f6da4a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5b37cd0e4504582981d3f91cb87c2d0" is_pre_election: true
I20260810 18:49:13.368650 30987 raft_consensus.cc:2454] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P d5b37cd0e4504582981d3f91cb87c2d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8f7e4fbfccc4de7a7bc51d60f6da4a9 in term 0.
I20260810 18:49:13.368829 31227 leader_election.cc:302] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P f8f7e4fbfccc4de7a7bc51d60f6da4a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aae9b99504854b59bd4ba8cc5ef5d891, f8f7e4fbfccc4de7a7bc51d60f6da4a9; no voters:
I20260810 18:49:13.369088 31393 raft_consensus.cc:2781] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P f8f7e4fbfccc4de7a7bc51d60f6da4a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:49:13.369174 31393 raft_consensus.cc:482] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P f8f7e4fbfccc4de7a7bc51d60f6da4a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:49:13.369217 31393 raft_consensus.cc:3037] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P f8f7e4fbfccc4de7a7bc51d60f6da4a9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:13.370186 31393 raft_consensus.cc:504] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P f8f7e4fbfccc4de7a7bc51d60f6da4a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8f7e4fbfccc4de7a7bc51d60f6da4a9" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45065 } } peers { permanent_uuid: "aae9b99504854b59bd4ba8cc5ef5d891" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 46881 } } peers { permanent_uuid: "d5b37cd0e4504582981d3f91cb87c2d0" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 34175 } }
I20260810 18:49:13.370491 31393 leader_election.cc:288] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P f8f7e4fbfccc4de7a7bc51d60f6da4a9 [CANDIDATE]: Term 1 election: Requested vote from peers aae9b99504854b59bd4ba8cc5ef5d891 (127.18.168.130:46881), d5b37cd0e4504582981d3f91cb87c2d0 (127.18.168.129:34175)
I20260810 18:49:13.370843 31151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f6c7a351b0f4757b24a5e98fa0d4b09" candidate_uuid: "f8f7e4fbfccc4de7a7bc51d60f6da4a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aae9b99504854b59bd4ba8cc5ef5d891"
I20260810 18:49:13.370895 30987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f6c7a351b0f4757b24a5e98fa0d4b09" candidate_uuid: "f8f7e4fbfccc4de7a7bc51d60f6da4a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5b37cd0e4504582981d3f91cb87c2d0"
I20260810 18:49:13.370982 31151 raft_consensus.cc:3037] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P aae9b99504854b59bd4ba8cc5ef5d891 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:13.371019 30987 raft_consensus.cc:3037] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P d5b37cd0e4504582981d3f91cb87c2d0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:13.371975 30987 raft_consensus.cc:2454] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P d5b37cd0e4504582981d3f91cb87c2d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8f7e4fbfccc4de7a7bc51d60f6da4a9 in term 1.
I20260810 18:49:13.371985 31151 raft_consensus.cc:2454] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P aae9b99504854b59bd4ba8cc5ef5d891 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8f7e4fbfccc4de7a7bc51d60f6da4a9 in term 1.
I20260810 18:49:13.372226 31227 leader_election.cc:302] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P f8f7e4fbfccc4de7a7bc51d60f6da4a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d5b37cd0e4504582981d3f91cb87c2d0, f8f7e4fbfccc4de7a7bc51d60f6da4a9; no voters:
I20260810 18:49:13.372390 31393 raft_consensus.cc:2781] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P f8f7e4fbfccc4de7a7bc51d60f6da4a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:49:13.372601 31393 raft_consensus.cc:686] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P f8f7e4fbfccc4de7a7bc51d60f6da4a9 [term 1 LEADER]: Becoming Leader. State: Replica: f8f7e4fbfccc4de7a7bc51d60f6da4a9, State: Running, Role: LEADER
I20260810 18:49:13.373116 31393 consensus_queue.cc:237] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P f8f7e4fbfccc4de7a7bc51d60f6da4a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8f7e4fbfccc4de7a7bc51d60f6da4a9" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45065 } } peers { permanent_uuid: "aae9b99504854b59bd4ba8cc5ef5d891" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 46881 } } peers { permanent_uuid: "d5b37cd0e4504582981d3f91cb87c2d0" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 34175 } }
I20260810 18:49:13.376013 30808 catalog_manager.cc:5495] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P f8f7e4fbfccc4de7a7bc51d60f6da4a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to f8f7e4fbfccc4de7a7bc51d60f6da4a9 (127.18.168.131). New cstate: current_term: 1 leader_uuid: "f8f7e4fbfccc4de7a7bc51d60f6da4a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8f7e4fbfccc4de7a7bc51d60f6da4a9" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aae9b99504854b59bd4ba8cc5ef5d891" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 46881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5b37cd0e4504582981d3f91cb87c2d0" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 34175 } health_report { overall_health: UNKNOWN } } }
I20260810 18:49:13.386706 31382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:13.383157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59372 (local address 127.18.168.131:45065)
0810 18:49:13.383279 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:13.383283 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:13.383295 (+ 12us) server_negotiation.cc:407] Connection header received
0810 18:49:13.383401 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:13.383403 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:13.383450 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:13.383549 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:49:13.383756 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.384409 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.385298 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.385506 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.385594 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:13.385866 (+ 272us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:49:13.386309 (+ 443us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:49:13.386397 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:13.386464 (+ 67us) server_negotiation.cc:299] Negotiation successful
0810 18:49:13.386509 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 18:49:13.392493 30987 raft_consensus.cc:1261] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P d5b37cd0e4504582981d3f91cb87c2d0 [term 1 FOLLOWER]: Refusing update from remote peer f8f7e4fbfccc4de7a7bc51d60f6da4a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:49:13.392493 31151 raft_consensus.cc:1261] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P aae9b99504854b59bd4ba8cc5ef5d891 [term 1 FOLLOWER]: Refusing update from remote peer f8f7e4fbfccc4de7a7bc51d60f6da4a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:49:13.393232 31393 consensus_queue.cc:1035] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P f8f7e4fbfccc4de7a7bc51d60f6da4a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aae9b99504854b59bd4ba8cc5ef5d891" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 46881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:49:13.393410 31397 consensus_queue.cc:1035] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P f8f7e4fbfccc4de7a7bc51d60f6da4a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5b37cd0e4504582981d3f91cb87c2d0" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 34175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:49:13.400175 31401 mvcc.cc:205] Tried to move back new op lower bound from 7317044237888241664 to 7317044237818757120. Current Snapshot: MvccSnapshot[applied={T|T < 7317044237888241664}]
I20260810 18:49:13.401528 31403 mvcc.cc:205] Tried to move back new op lower bound from 7317044237888241664 to 7317044237818757120. Current Snapshot: MvccSnapshot[applied={T|T < 7317044237888241664}]
I20260810 18:49:13.403304 31402 mvcc.cc:205] Tried to move back new op lower bound from 7317044237888241664 to 7317044237818757120. Current Snapshot: MvccSnapshot[applied={T|T < 7317044237888241664}]
I20260810 18:49:13.408869 30808 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:49:13.409004 30808 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:49:13.411375 30808 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 45cf0694830e43fba2ddfffd6d51b2d1: Sending DeleteTablet for 3 replicas of tablet 2f6c7a351b0f4757b24a5e98fa0d4b09
I20260810 18:49:13.412163 31122 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f6c7a351b0f4757b24a5e98fa0d4b09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:49:13 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:47874
I20260810 18:49:13.412175 31278 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f6c7a351b0f4757b24a5e98fa0d4b09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:49:13 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:59358
I20260810 18:49:13.412220 30963 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f6c7a351b0f4757b24a5e98fa0d4b09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:49:13 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:40578
I20260810 18:49:13.412645 31413 tablet_replica.cc:330] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P aae9b99504854b59bd4ba8cc5ef5d891: stopping tablet replica
I20260810 18:49:13.412801 31415 tablet_replica.cc:330] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P f8f7e4fbfccc4de7a7bc51d60f6da4a9: stopping tablet replica
I20260810 18:49:13.412971 31413 raft_consensus.cc:2229] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P aae9b99504854b59bd4ba8cc5ef5d891 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:49:13.413015 31415 raft_consensus.cc:2229] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P f8f7e4fbfccc4de7a7bc51d60f6da4a9 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:49:13.413297 31413 raft_consensus.cc:2258] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P aae9b99504854b59bd4ba8cc5ef5d891 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:49:13.413324 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 30887
I20260810 18:49:13.413338 31415 raft_consensus.cc:2258] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P f8f7e4fbfccc4de7a7bc51d60f6da4a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:49:13.415482 31413 ts_tablet_manager.cc:1905] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P aae9b99504854b59bd4ba8cc5ef5d891: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:49:13.419538 31413 ts_tablet_manager.cc:1918] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P aae9b99504854b59bd4ba8cc5ef5d891: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:49:13.419721 31413 log.cc:1196] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P aae9b99504854b59bd4ba8cc5ef5d891: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-1/wal/wals/2f6c7a351b0f4757b24a5e98fa0d4b09
I20260810 18:49:13.420102 31415 ts_tablet_manager.cc:1905] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P f8f7e4fbfccc4de7a7bc51d60f6da4a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:49:13.420238 31413 ts_tablet_manager.cc:1939] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P aae9b99504854b59bd4ba8cc5ef5d891: Deleting consensus metadata
I20260810 18:49:13.421573 30793 catalog_manager.cc:4841] TS aae9b99504854b59bd4ba8cc5ef5d891 (127.18.168.130:46881): tablet 2f6c7a351b0f4757b24a5e98fa0d4b09 (table test-table [id=133d1bae13fa487db616997736e209b5]) successfully deleted
I20260810 18:49:13.423291 31415 ts_tablet_manager.cc:1918] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P f8f7e4fbfccc4de7a7bc51d60f6da4a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:49:13.423406 31415 log.cc:1196] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P f8f7e4fbfccc4de7a7bc51d60f6da4a9: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387687857264-19106-0/minicluster-data/ts-2/wal/wals/2f6c7a351b0f4757b24a5e98fa0d4b09
I20260810 18:49:13.423722 31415 ts_tablet_manager.cc:1939] T 2f6c7a351b0f4757b24a5e98fa0d4b09 P f8f7e4fbfccc4de7a7bc51d60f6da4a9: Deleting consensus metadata
I20260810 18:49:13.424665 30795 catalog_manager.cc:4841] TS f8f7e4fbfccc4de7a7bc51d60f6da4a9 (127.18.168.131:45065): tablet 2f6c7a351b0f4757b24a5e98fa0d4b09 (table test-table [id=133d1bae13fa487db616997736e209b5]) successfully deleted
W20260810 18:49:13.429198 30793 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.168.129:34175 (error 108)
I20260810 18:49:13.430028 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 31047
I20260810 18:49:13.430243 31384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:13.429905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:34175 (local address 127.0.0.1:40580)
0810 18:49:13.430032 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:13.430137 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.129:34175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260810 18:49:13.431676 30793 catalog_manager.cc:4584] TS d5b37cd0e4504582981d3f91cb87c2d0 (127.18.168.129:34175): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2f6c7a351b0f4757b24a5e98fa0d4b09: Network error: Client connection negotiation failed: client connection to 127.18.168.129:34175: connect: Connection refused (error 111)
I20260810 18:49:13.437462 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 31207
I20260810 18:49:13.446578 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 30775
2026-08-10T18:49:13Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2325 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:49:13 dist-test-slave-cbsf krb5kdc[31421](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:49:13 dist-test-slave-cbsf krb5kdc[31421](info): set up 2 sockets
Aug 10 18:49:13 dist-test-slave-cbsf krb5kdc[31421](info): commencing operation
krb5kdc: starting...
W20260810 18:49:14.549132 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:49:14 dist-test-slave-cbsf krb5kdc[31421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:49:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:49:14Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.18.168.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.18.168.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.18.168.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/krb5kdc/rangeradmin_127.18.168.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.18.168.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/krb5kdc/rangeradmin_127.18.168.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.18.168.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.18.168.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.18.168.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/krb5kdc/rangerlookup_127.18.168.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.18.168.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/krb5kdc/rangerlookup_127.18.168.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.168.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.18.168.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.168.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/krb5kdc/HTTP_127.18.168.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.168.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/krb5kdc/HTTP_127.18.168.148@KRBTEST.COM.keytab.
I20260810 18:49:14.717991 19106 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task1VMCzb/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/postgres -l logfile start
2026-08-10 18:49:17.236 UTC [31460] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 18:49:17.237 UTC [31460] LOG: listening on IPv4 address "127.18.168.148", port 48905
2026-08-10 18:49:17.241 UTC [31460] LOG: listening on Unix socket "/tmp/.s.PGSQL.48905"
2026-08-10 18:49:17.264 UTC [31463] LOG: database system was shut down at 2026-08-10 18:49:15 UTC
2026-08-10 18:49:17.268 UTC [31460] LOG: database system is ready to accept connections
I20260810 18:49:18.285367 19106 mini_postgres.cc:96] Postgres bound to 48905
2026-08-10 18:49:18.293 UTC [31472] FATAL: database "slave" does not exist
127.18.168.148:48905 - accepting connections
I20260810 18:49:18.294693 19106 mini_ranger.cc:161] Starting Ranger...
I20260810 18:49:18.310531 19106 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 18:49:18.762378 19106 mini_ranger.cc:87] Created ranger Postgres database
I20260810 18:49:18.762490 19106 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ranger-admin
2026-08-10 18:49:19,078 [I] DB FLAVOR :POSTGRES
2026-08-10 18:49:19,079 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 18:49:19,079 [I] Checking connection..
2026-08-10 18:49:19,079 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:48905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 18:49:19,365 [I] Checking connection passed.
2026-08-10 18:49:19,366 [I] --------- Verifying version history table ---------
2026-08-10 18:49:19,366 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:48905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 18:49:19,637 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 18:49:19,637 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:48905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 18:49:19,890 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 18:49:19,891 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 18:49:19,891 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:48905/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 18:49:20.162 UTC [31546] WARNING: there is no transaction in progress
2026-08-10 18:49:20,169 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 18:49:20,170 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:48905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 18:49:20,402 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 18:49:20,402 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 18:49:20,402 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:48905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 18:49:20,634 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:48905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 18:49:20,893 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:48905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-cbsf.c.gcp-upstream.internal','N') ;"
2026-08-10 18:49:21,134 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:48905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 18:49:21,399 [I] Table x_portal_user does not exist in database ranger
2026-08-10 18:49:21,399 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:48905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 18:49:21,655 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 18:49:21,655 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:48905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 18:49:21,890 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:48905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 18:49:22,162 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 18:49:22,162 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:48905/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 18:49:23.134 UTC [31678] WARNING: there is no transaction in progress
2026-08-10 18:49:23.150 UTC [31678] WARNING: there is no transaction in progress
2026-08-10 18:49:23.166 UTC [31678] WARNING: there is no transaction in progress
2026-08-10 18:49:23.181 UTC [31678] WARNING: there is no transaction in progress
2026-08-10 18:49:23.198 UTC [31678] WARNING: there is no transaction in progress
2026-08-10 18:49:23.215 UTC [31678] WARNING: there is no transaction in progress
2026-08-10 18:49:23.269 UTC [31678] WARNING: there is no transaction in progress
2026-08-10 18:49:23.276 UTC [31678] WARNING: there is no transaction in progress
2026-08-10 18:49:23.290 UTC [31678] WARNING: there is no transaction in progress
2026-08-10 18:49:23.300 UTC [31678] WARNING: there is no transaction in progress
2026-08-10 18:49:23.310 UTC [31678] WARNING: there is no transaction in progress
2026-08-10 18:49:23.325 UTC [31678] WARNING: there is no transaction in progress
2026-08-10 18:49:23.334 UTC [31678] WARNING: there is no transaction in progress
2026-08-10 18:49:23.341 UTC [31678] WARNING: there is no transaction in progress
2026-08-10 18:49:23.348 UTC [31678] WARNING: there is no transaction in progress
2026-08-10 18:49:23.354 UTC [31678] WARNING: there is no transaction in progress
2026-08-10 18:49:23,968 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 18:49:23,969 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:48905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 18:49:24,225 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 18:49:24,226 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:48905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 18:49:24,473 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 18:49:24,474 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:48905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 18:49:24,736 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 18:49:24,736 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:48905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 18:49:24,998 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:48905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 18:49:25,259 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:48905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-cbsf.c.gcp-upstream.internal';"
2026-08-10 18:49:25,503 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 18:49:25,503 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:48905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 18:49:25,759 [I] DB_PATCHES have already been applied
I20260810 18:49:25.764990 19106 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ranger-admin:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task1VMCzb/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 18:49:25.765059 19106 mini_ranger.cc:193] Using host: 127.18.168.148
I20260810 18:49:25.770035 19106 mini_ranger.cc:227] Ranger admin URL: http://127.18.168.148:43513
Aug 10, 2026 6:49:26 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 6:49:26 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 6:49:26 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 6:49:26 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 6:49:26 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 6:49:26 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 6:49:26 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 18:49:27.064653 19106 mini_ranger.cc:160] Time spent starting Ranger: real 8.770s user 0.005s sys 0.008s
Aug 10, 2026 6:49:47 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 6:49:47 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 10, 2026 6:49:47 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 6:49:47 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 6:49:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 6:49:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 6:49:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 6:49:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 6:49:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 6:49:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 6:49:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 6:49:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 6:49:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 6:49:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 6:49:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 6:49:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 6:49:47 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 6:49:48 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 18:49:49.136410 19106 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.18.168.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.18.168.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.18.168.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/krb5kdc/rangerkms_127.18.168.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.18.168.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/krb5kdc/rangerkms_127.18.168.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.168.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.18.168.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.168.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/krb5kdc/HTTP_127.18.168.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.168.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/krb5kdc/HTTP_127.18.168.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 18:49:49 dist-test-slave-cbsf krb5kdc[31421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387789, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 18:49:49.244647 19106 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 18:49:49.257815 19106 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 18:49:50.096191 19106 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 18:49:50.096295 19106 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ranger-kms
I20260810 18:49:50.096407 19106 mini_ranger_kms.cc:227] Using postgres at 127.18.168.148:48905
2026-08-10 18:49:51,383 [I] DB FLAVOR :POSTGRES
2026-08-10 18:49:51,383 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 18:49:51,384 [I] Checking connection
2026-08-10 18:49:51,384 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:48905/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 18:49:51,767 [I] connection success
2026-08-10 18:49:51,767 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 18:49:51,767 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 18:49:51,767 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:48905/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 18:49:52,020 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 18:49:52,020 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 18:49:52,020 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 18:49:52,020 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:48905/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 18:49:52,279 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 18:49:52.754601 19106 mini_ranger_kms.cc:310] Created kms service
I20260810 18:49:52.955209 19106 mini_ranger_kms.cc:326] Created kudu user
I20260810 18:49:53.004585 19106 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 18:49:53 dist-test-slave-cbsf krb5kdc[31421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387789, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.18.168.148@KRBTEST.COM
I20260810 18:49:53.438117 19106 mini_ranger_kms.cc:384] Added ranger policy
I20260810 18:49:53.438283 19106 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ranger-kms:/tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 18:49:53.438304 19106 mini_ranger_kms.cc:240] Using host: 127.18.168.148
I20260810 18:49:53.443063 19106 mini_ranger_kms.cc:276] Ranger KMS PID: 31885
I20260810 18:49:53.443199 19106 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.18.168.148:40383
Aug 10, 2026 6:49:54 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 6:49:54 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 6:49:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 6:49:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 6:49:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 6:49:55 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260810 18:49:55.954095 19106 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.709s user 0.023s sys 0.982s
I20260810 18:49:55.954247 19106 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 18:49:55.954391 19106 mini_ranger_kms.cc:401] 127.18.168.148:40383/kms/v1/keys
Aug 10 18:49:56 dist-test-slave-cbsf krb5kdc[31421](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786387796, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.168.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 10 18:49:56 dist-test-slave-cbsf krb5kdc[31421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387796, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.168.148@KRBTEST.COM for HTTP/127.18.168.148@KRBTEST.COM
Aug 10 18:49:57 dist-test-slave-cbsf krb5kdc[31421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387796, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.168.148@KRBTEST.COM for HTTP/127.18.168.148@KRBTEST.COM
Aug 10 18:49:59 dist-test-slave-cbsf krb5kdc[31421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.18.168.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
I20260810 18:49:59.871608 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:35961
--webserver_interface=127.18.168.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:38677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:35961
--ranger_config_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.168.148:40383/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:50:00.082964 31946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:50:00.083276 31946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:50:00.083356 31946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:50:00.087930 31946 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:50:00.088014 31946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:50:00.088055 31946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:50:00.088117 31946 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:50:00.088160 31946 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:50:00.094662 31946 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:38677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/master-0/wal
--ranger_kms_url=127.18.168.148:40383/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:35961
--ranger_config_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:35961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:50:00.095789 31946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:50:00.096947 31946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:50:00.107556 31953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:50:00.107579 31952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:50:00.107604 31956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:50:00.107903 31946 server_base.cc:1034] running on GCE node
I20260810 18:50:00.108476 31946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:50:00.109630 31946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:50:00.110833 31946 hybrid_clock.cc:648] HybridClock initialized: now 1786387800110809 us; error 36 us; skew 500 ppm
Aug 10 18:50:00 dist-test-slave-cbsf krb5kdc[31421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387800, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:50:00.114251 31946 init.cc:377] Logged in from keytab as kudu/127.18.168.190@KRBTEST.COM (short username kudu)
I20260810 18:50:00.115165 31946 webserver.cc:460] Webserver started at http://127.18.168.190:40065/ using document root <none> and password file <none>
I20260810 18:50:00.115739 31946 fs_manager.cc:359] Metadata directory not provided
I20260810 18:50:00.115823 31946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:50:00.116047 31946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 18:50:00 dist-test-slave-cbsf krb5kdc[31421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387800, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for HTTP/127.18.168.148@KRBTEST.COM
I20260810 18:50:00.182346 31946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "ae543986c9694a3e9e5bbc0580a74307"
format_stamp: "Formatted at 2026-08-10 18:50:00 on dist-test-slave-cbsf"
server_key: "a1cba2961738ad9319e40315386781b7"
server_key_iv: "f60056dc93f20a414b8360405aab1363"
server_key_version: "kuduclusterkey@0"
I20260810 18:50:00.183284 31946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "ae543986c9694a3e9e5bbc0580a74307"
format_stamp: "Formatted at 2026-08-10 18:50:00 on dist-test-slave-cbsf"
server_key: "a1cba2961738ad9319e40315386781b7"
server_key_iv: "f60056dc93f20a414b8360405aab1363"
server_key_version: "kuduclusterkey@0"
I20260810 18:50:00.189019 31946 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260810 18:50:00.207720 31966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:00.209388 31946 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:50:00.209546 31946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "ae543986c9694a3e9e5bbc0580a74307"
format_stamp: "Formatted at 2026-08-10 18:50:00 on dist-test-slave-cbsf"
server_key: "a1cba2961738ad9319e40315386781b7"
server_key_iv: "f60056dc93f20a414b8360405aab1363"
server_key_version: "kuduclusterkey@0"
I20260810 18:50:00.209686 31946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:50:00.225150 31946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:50:00.228178 31946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:50:00.228384 31946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:50:00.247177 31946 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:35961
I20260810 18:50:00.247294 32029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:35961 every 8 connection(s)
I20260810 18:50:00.248565 31946 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:50:00.251379 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 31946
I20260810 18:50:00.251502 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:50:00.251814 19106 external_mini_cluster.cc:1428] Setting key 8be188bc3d1287b933ce293f124dab9d
I20260810 18:50:00.251924 32030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:50:00.260027 32030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae543986c9694a3e9e5bbc0580a74307: Bootstrap starting.
I20260810 18:50:00.262372 32030 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae543986c9694a3e9e5bbc0580a74307: Neither blocks nor log segments found. Creating new log.
Aug 10 18:50:00 dist-test-slave-cbsf krb5kdc[31421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:50:00.263341 32030 log.cc:824] T 00000000000000000000000000000000 P ae543986c9694a3e9e5bbc0580a74307: Log is configured to *not* fsync() on all Append() calls
I20260810 18:50:00.265784 32030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae543986c9694a3e9e5bbc0580a74307: No bootstrap required, opened a new log
I20260810 18:50:00.269444 32034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:00.253317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35798 (local address 127.18.168.190:35961)
0810 18:50:00.254006 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:00.254040 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:00.254084 (+ 44us) server_negotiation.cc:407] Connection header received
0810 18:50:00.255172 (+ 1088us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:00.255198 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:00.255735 (+ 537us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:00.256138 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:50:00.256981 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:00.258512 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:00.259479 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:00.260200 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:00.260364 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:00.263608 (+ 3244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:50:00.263613 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:50:00.263626 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:50:00.263657 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:50:00.266364 (+ 2707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:00.266873 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:00.266878 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:00.266882 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:00.266991 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:00.267327 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:00.267329 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:00.267330 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:00.267734 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:50:00.267882 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:00.268237 (+ 355us) server_negotiation.cc:299] Negotiation successful
0810 18:50:00.268553 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":343,"threads_started":1}
I20260810 18:50:00.269585 32030 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae543986c9694a3e9e5bbc0580a74307 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae543986c9694a3e9e5bbc0580a74307" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 35961 } }
I20260810 18:50:00.269879 32030 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae543986c9694a3e9e5bbc0580a74307 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:50:00.269974 32030 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae543986c9694a3e9e5bbc0580a74307 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae543986c9694a3e9e5bbc0580a74307, State: Initialized, Role: FOLLOWER
I20260810 18:50:00.270447 32030 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae543986c9694a3e9e5bbc0580a74307 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae543986c9694a3e9e5bbc0580a74307" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 35961 } }
I20260810 18:50:00.270614 32030 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae543986c9694a3e9e5bbc0580a74307 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:50:00.270689 32030 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae543986c9694a3e9e5bbc0580a74307 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:50:00.270785 32030 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae543986c9694a3e9e5bbc0580a74307 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:50:00.271881 32030 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae543986c9694a3e9e5bbc0580a74307 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae543986c9694a3e9e5bbc0580a74307" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 35961 } }
I20260810 18:50:00.272223 32030 leader_election.cc:302] T 00000000000000000000000000000000 P ae543986c9694a3e9e5bbc0580a74307 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae543986c9694a3e9e5bbc0580a74307; no voters:
I20260810 18:50:00.272537 32030 leader_election.cc:288] T 00000000000000000000000000000000 P ae543986c9694a3e9e5bbc0580a74307 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:50:00.272894 32036 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae543986c9694a3e9e5bbc0580a74307 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:50:00.273190 32036 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae543986c9694a3e9e5bbc0580a74307 [term 1 LEADER]: Becoming Leader. State: Replica: ae543986c9694a3e9e5bbc0580a74307, State: Running, Role: LEADER
I20260810 18:50:00.273553 32036 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae543986c9694a3e9e5bbc0580a74307 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae543986c9694a3e9e5bbc0580a74307" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 35961 } }
I20260810 18:50:00.274024 32030 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae543986c9694a3e9e5bbc0580a74307 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:50:00.275486 32037 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae543986c9694a3e9e5bbc0580a74307 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae543986c9694a3e9e5bbc0580a74307" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae543986c9694a3e9e5bbc0580a74307" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 35961 } } }
I20260810 18:50:00.275524 32038 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae543986c9694a3e9e5bbc0580a74307 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae543986c9694a3e9e5bbc0580a74307. Latest consensus state: current_term: 1 leader_uuid: "ae543986c9694a3e9e5bbc0580a74307" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae543986c9694a3e9e5bbc0580a74307" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 35961 } } }
I20260810 18:50:00.275648 32037 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae543986c9694a3e9e5bbc0580a74307 [sys.catalog]: This master's current role is: LEADER
I20260810 18:50:00.275660 32038 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae543986c9694a3e9e5bbc0580a74307 [sys.catalog]: This master's current role is: LEADER
I20260810 18:50:00.276595 32041 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:50:00.280208 32041 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:50:00.280771 32030 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 18:50:00.298409 32041 catalog_manager.cc:1347] Generated new cluster ID: 72db9f3545664397b3f565681a7ced1f
I20260810 18:50:00.298547 32041 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:50:00.310314 32041 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:50:00.311293 32041 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:50:00.318643 32041 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae543986c9694a3e9e5bbc0580a74307: Generated new TSK 0
I20260810 18:50:00.319387 32041 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.168.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab.
Entry for principal kudu/127.18.168.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab.
I20260810 18:50:01.332279 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.168.148:40383/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.129:0
--local_ip_for_outbound_sockets=127.18.168.129
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:35961
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:38677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:50:01.542989 32082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:50:01.543455 32082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:50:01.543596 32082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:50:01.553265 32082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:50:01.553428 32082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:50:01.553628 32082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.129
I20260810 18:50:01.563249 32082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:38677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.18.168.148:40383/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:35961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.129
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:50:01.564819 32082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:50:01.566386 32082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:50:01.579733 32089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:50:01.579704 32091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:50:01.579730 32087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:50:01.580374 32082 server_base.cc:1034] running on GCE node
I20260810 18:50:01.580912 32082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:50:01.581537 32082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:50:01.582734 32082 hybrid_clock.cc:648] HybridClock initialized: now 1786387801582708 us; error 63 us; skew 500 ppm
Aug 10 18:50:01 dist-test-slave-cbsf krb5kdc[31421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387801, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:50:01.586014 32082 init.cc:377] Logged in from keytab as kudu/127.18.168.129@KRBTEST.COM (short username kudu)
I20260810 18:50:01.587378 32082 webserver.cc:460] Webserver started at http://127.18.168.129:37083/ using document root <none> and password file <none>
I20260810 18:50:01.588277 32082 fs_manager.cc:359] Metadata directory not provided
I20260810 18:50:01.588383 32082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:50:01.588632 32082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 18:50:01 dist-test-slave-cbsf krb5kdc[31421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387801, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for HTTP/127.18.168.148@KRBTEST.COM
I20260810 18:50:01.605300 32082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-0/data/instance:
uuid: "39aa5565bccb4e30a4c36b2b2cc98931"
format_stamp: "Formatted at 2026-08-10 18:50:01 on dist-test-slave-cbsf"
server_key: "a41e0cad95dc3352bfd0d22555b210bf"
server_key_iv: "07892f41e8bd39a238e7f7120f53b6d1"
server_key_version: "kuduclusterkey@0"
I20260810 18:50:01.606215 32082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance:
uuid: "39aa5565bccb4e30a4c36b2b2cc98931"
format_stamp: "Formatted at 2026-08-10 18:50:01 on dist-test-slave-cbsf"
server_key: "a41e0cad95dc3352bfd0d22555b210bf"
server_key_iv: "07892f41e8bd39a238e7f7120f53b6d1"
server_key_version: "kuduclusterkey@0"
I20260810 18:50:01.615708 32082 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.004s sys 0.000s
I20260810 18:50:01.631633 32103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:01.633229 32082 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 18:50:01.633350 32082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-0/wal
uuid: "39aa5565bccb4e30a4c36b2b2cc98931"
format_stamp: "Formatted at 2026-08-10 18:50:01 on dist-test-slave-cbsf"
server_key: "a41e0cad95dc3352bfd0d22555b210bf"
server_key_iv: "07892f41e8bd39a238e7f7120f53b6d1"
server_key_version: "kuduclusterkey@0"
I20260810 18:50:01.633464 32082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:50:01.659894 32082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:50:01.664464 32082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:50:01.664746 32082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:50:01.665773 32082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:50:01.667184 32082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:50:01.667272 32082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:01.667330 32082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:50:01.667604 32082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:01.695464 32082 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.129:40357
I20260810 18:50:01.695732 32235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.129:40357 every 8 connection(s)
I20260810 18:50:01.697129 32082 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
I20260810 18:50:01.702550 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 32082
I20260810 18:50:01.702735 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance
I20260810 18:50:01.703135 19106 external_mini_cluster.cc:1428] Setting key 8e342687bff6197895faf80f7f983a95
Aug 10 18:50:01 dist-test-slave-cbsf krb5kdc[31421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387801, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:50:01.718747 32238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:01.699949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:50585 (local address 127.18.168.190:35961)
0810 18:50:01.700259 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:01.700263 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:01.701448 (+ 1185us) server_negotiation.cc:407] Connection header received
0810 18:50:01.702799 (+ 1351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:01.702803 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:01.702872 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:01.702961 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:50:01.704792 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:01.705754 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:01.708242 (+ 2488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:01.708669 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:01.708838 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:01.712976 (+ 4138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:50:01.712983 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:50:01.712989 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:50:01.713024 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:50:01.715345 (+ 2321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:01.716223 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:01.716228 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:01.716230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:01.716376 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:01.716969 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:01.716974 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:01.716976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:01.717310 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:50:01.717429 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:01.718333 (+ 904us) server_negotiation.cc:299] Negotiation successful
0810 18:50:01.718498 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":135,"threads_started":1}
WARNING: no policy specified for kudu/127.18.168.130@KRBTEST.COM; defaulting to no policy
I20260810 18:50:01.720194 32240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:01.700514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:35961 (local address 127.18.168.129:50585)
0810 18:50:01.701288 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:01.701332 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 18:50:01.702568 (+ 1236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:50:01.703143 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:50:01.703150 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:50:01.703739 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:50:01.704598 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:01.704619 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:01.705923 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:01.705930 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:01.708005 (+ 2075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:01.708017 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:01.708901 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:01.708905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:01.709264 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:01.710188 (+ 924us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:50:01.710215 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:50:01.712730 (+ 2515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:50:01.715487 (+ 2757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:01.715494 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:01.715531 (+ 37us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:01.716068 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:01.716506 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:01.716510 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:01.716512 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:01.716811 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:01.717483 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:50:01.717490 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:50:01.717892 (+ 402us) client_negotiation.cc:769] Sending connection context
0810 18:50:01.718241 (+ 349us) client_negotiation.cc:240] Negotiation successful
0810 18:50:01.718741 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":358,"threads_started":1}
I20260810 18:50:01.721877 32236 heartbeater.cc:347] Connected to a master server at 127.18.168.190:35961
I20260810 18:50:01.722214 32236 heartbeater.cc:464] Registering TS with master...
I20260810 18:50:01.722808 32236 heartbeater.cc:511] Master 127.18.168.190:35961 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.130@KRBTEST.COM" created.
I20260810 18:50:01.724684 31984 ts_manager.cc:194] Registered new tserver with Master: 39aa5565bccb4e30a4c36b2b2cc98931 (127.18.168.129:40357)
I20260810 18:50:01.726643 31984 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.129@KRBTEST.COM'} at 127.18.168.129:50585
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab.
Entry for principal kudu/127.18.168.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab.
I20260810 18:50:01.752259 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.168.148:40383/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.130:0
--local_ip_for_outbound_sockets=127.18.168.130
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:35961
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:38677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 18:50:01 dist-test-slave-cbsf krb5kdc[31421](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786387801, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260810 18:50:01.980892 32243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:50:01.981240 32243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:50:01.981307 32243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:50:01.987221 32243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:50:01.987349 32243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:50:01.987466 32243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.130
I20260810 18:50:01.995894 32243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:38677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.18.168.148:40383/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:35961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.130
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:50:01.997241 32243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:50:01.998560 32243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:50:02.013898 32252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:50:02.013888 32251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:50:02.014037 32255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:50:02.016485 32243 server_base.cc:1034] running on GCE node
I20260810 18:50:02.017117 32243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:50:02.018093 32243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:50:02.019346 32243 hybrid_clock.cc:648] HybridClock initialized: now 1786387802019237 us; error 116 us; skew 500 ppm
Aug 10 18:50:02 dist-test-slave-cbsf krb5kdc[31421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387802, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:50:02.029021 32243 init.cc:377] Logged in from keytab as kudu/127.18.168.130@KRBTEST.COM (short username kudu)
I20260810 18:50:02.030041 32243 webserver.cc:460] Webserver started at http://127.18.168.130:39177/ using document root <none> and password file <none>
I20260810 18:50:02.030678 32243 fs_manager.cc:359] Metadata directory not provided
I20260810 18:50:02.030774 32243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:50:02.031006 32243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 18:50:02 dist-test-slave-cbsf krb5kdc[31421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387802, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for HTTP/127.18.168.148@KRBTEST.COM
I20260810 18:50:02.050851 32243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-1/data/instance:
uuid: "c7e9fde0f7f142eebe48fbfa7585fdc0"
format_stamp: "Formatted at 2026-08-10 18:50:02 on dist-test-slave-cbsf"
server_key: "b5df9daccd9d4c08fe0bb21f3c6f86fb"
server_key_iv: "0f2a109b6667f2754a3fcdde7f980bf3"
server_key_version: "kuduclusterkey@0"
I20260810 18:50:02.051937 32243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance:
uuid: "c7e9fde0f7f142eebe48fbfa7585fdc0"
format_stamp: "Formatted at 2026-08-10 18:50:02 on dist-test-slave-cbsf"
server_key: "b5df9daccd9d4c08fe0bb21f3c6f86fb"
server_key_iv: "0f2a109b6667f2754a3fcdde7f980bf3"
server_key_version: "kuduclusterkey@0"
I20260810 18:50:02.058892 32243 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260810 18:50:02.089242 32265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:02.090991 32243 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260810 18:50:02.091122 32243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-1/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-1/wal
uuid: "c7e9fde0f7f142eebe48fbfa7585fdc0"
format_stamp: "Formatted at 2026-08-10 18:50:02 on dist-test-slave-cbsf"
server_key: "b5df9daccd9d4c08fe0bb21f3c6f86fb"
server_key_iv: "0f2a109b6667f2754a3fcdde7f980bf3"
server_key_version: "kuduclusterkey@0"
I20260810 18:50:02.091250 32243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:50:02.118914 32243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:50:02.123392 32243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:50:02.123641 32243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:50:02.124550 32243 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:50:02.125815 32243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:50:02.125913 32243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:02.126032 32243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:50:02.126083 32243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:02.153748 32243 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.130:36171
I20260810 18:50:02.153802 32398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.130:36171 every 8 connection(s)
I20260810 18:50:02.155462 32243 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
I20260810 18:50:02.163970 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 32243
I20260810 18:50:02.164096 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance
I20260810 18:50:02.164404 19106 external_mini_cluster.cc:1428] Setting key 9ff5b786e7b76622d42198351645acd1
Aug 10 18:50:02 dist-test-slave-cbsf krb5kdc[31421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387802, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:50:02.179040 32238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:02.158106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:41727 (local address 127.18.168.190:35961)
0810 18:50:02.158268 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:02.158275 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:02.159631 (+ 1356us) server_negotiation.cc:407] Connection header received
0810 18:50:02.161601 (+ 1970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:02.161619 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:02.161695 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:02.161782 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:50:02.164306 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:02.165233 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:02.167489 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:02.167886 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:02.168104 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:02.172584 (+ 4480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:50:02.172589 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:50:02.172592 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:50:02.172625 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:50:02.174900 (+ 2275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:02.175755 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:02.175759 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:02.175761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:02.175866 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:02.177097 (+ 1231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:02.177101 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:02.177103 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:02.177460 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:50:02.177559 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:02.178631 (+ 1072us) server_negotiation.cc:299] Negotiation successful
0810 18:50:02.178796 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 18:50:02.181272 32401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:02.158576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:35961 (local address 127.18.168.130:41727)
0810 18:50:02.159424 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:02.159489 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0810 18:50:02.161291 (+ 1802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:50:02.162305 (+ 1014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:50:02.162315 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:50:02.162989 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:50:02.164061 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:02.164085 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:02.165505 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:02.165509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:02.167279 (+ 1770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:02.167292 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:02.168381 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:02.168386 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:02.168731 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:02.169832 (+ 1101us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:50:02.169854 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:50:02.172255 (+ 2401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:50:02.175087 (+ 2832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:02.175094 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:02.175111 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:02.175535 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:02.176133 (+ 598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:02.176137 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:02.176139 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:02.176626 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:02.177704 (+ 1078us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:50:02.177712 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:50:02.178177 (+ 465us) client_negotiation.cc:769] Sending connection context
0810 18:50:02.179277 (+ 1100us) client_negotiation.cc:240] Negotiation successful
0810 18:50:02.179708 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":268,"threads_started":1}
WARNING: no policy specified for kudu/127.18.168.131@KRBTEST.COM; defaulting to no policy
I20260810 18:50:02.183130 32399 heartbeater.cc:347] Connected to a master server at 127.18.168.190:35961
I20260810 18:50:02.183467 32399 heartbeater.cc:464] Registering TS with master...
I20260810 18:50:02.184211 32399 heartbeater.cc:511] Master 127.18.168.190:35961 requested a full tablet report, sending...
I20260810 18:50:02.185863 31984 ts_manager.cc:194] Registered new tserver with Master: c7e9fde0f7f142eebe48fbfa7585fdc0 (127.18.168.130:36171)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.131@KRBTEST.COM" created.
I20260810 18:50:02.186789 31984 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.130@KRBTEST.COM'} at 127.18.168.130:41727
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab.
Entry for principal kudu/127.18.168.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab.
I20260810 18:50:02.211721 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.168.148:40383/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.131:0
--local_ip_for_outbound_sockets=127.18.168.131
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:35961
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:38677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:50:02.440393 32404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:50:02.440697 32404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:50:02.440845 32404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:50:02.445504 32404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:50:02.445609 32404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:50:02.445719 32404 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.131
I20260810 18:50:02.451267 32404 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:38677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.18.168.148:40383/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:35961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.131
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:50:02.452463 32404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:50:02.453845 32404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:50:02.471309 32409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:50:02.471393 32410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:50:02.471650 32413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:50:02.473994 32404 server_base.cc:1034] running on GCE node
I20260810 18:50:02.474555 32404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:50:02.475452 32404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:50:02.476641 32404 hybrid_clock.cc:648] HybridClock initialized: now 1786387802476578 us; error 80 us; skew 500 ppm
Aug 10 18:50:02 dist-test-slave-cbsf krb5kdc[31421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387802, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:50:02.481756 32404 init.cc:377] Logged in from keytab as kudu/127.18.168.131@KRBTEST.COM (short username kudu)
I20260810 18:50:02.482945 32404 webserver.cc:460] Webserver started at http://127.18.168.131:37487/ using document root <none> and password file <none>
I20260810 18:50:02.483657 32404 fs_manager.cc:359] Metadata directory not provided
I20260810 18:50:02.483788 32404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:50:02.484061 32404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 18:50:02 dist-test-slave-cbsf krb5kdc[31421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387802, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for HTTP/127.18.168.148@KRBTEST.COM
I20260810 18:50:02.505986 32404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-2/data/instance:
uuid: "a265e3ff08ab4dc8ac72a45b02156a5b"
format_stamp: "Formatted at 2026-08-10 18:50:02 on dist-test-slave-cbsf"
server_key: "32f43d50b0282a5dbafe5e9fa226efb0"
server_key_iv: "6202603d5a437085d87bcb2e74defe76"
server_key_version: "kuduclusterkey@0"
I20260810 18:50:02.506898 32404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance:
uuid: "a265e3ff08ab4dc8ac72a45b02156a5b"
format_stamp: "Formatted at 2026-08-10 18:50:02 on dist-test-slave-cbsf"
server_key: "32f43d50b0282a5dbafe5e9fa226efb0"
server_key_iv: "6202603d5a437085d87bcb2e74defe76"
server_key_version: "kuduclusterkey@0"
I20260810 18:50:02.512961 32404 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260810 18:50:02.535753 32421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:02.537710 32404 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260810 18:50:02.537911 32404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-2/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-2/wal
uuid: "a265e3ff08ab4dc8ac72a45b02156a5b"
format_stamp: "Formatted at 2026-08-10 18:50:02 on dist-test-slave-cbsf"
server_key: "32f43d50b0282a5dbafe5e9fa226efb0"
server_key_iv: "6202603d5a437085d87bcb2e74defe76"
server_key_version: "kuduclusterkey@0"
I20260810 18:50:02.538080 32404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 10 18:50:02 dist-test-slave-cbsf krb5kdc[31421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387801, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for HTTP/127.18.168.148@KRBTEST.COM
I20260810 18:50:02.569464 32404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:50:02.573143 32404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:50:02.573413 32404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:50:02.574337 32404 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:50:02.575405 32404 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:50:02.575522 32404 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:02.575624 32404 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:50:02.575691 32404 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:02.610941 32404 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.131:34709
I20260810 18:50:02.611145 32556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.131:34709 every 8 connection(s)
I20260810 18:50:02.612313 32404 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
I20260810 18:50:02.619765 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 32404
I20260810 18:50:02.619908 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance
I20260810 18:50:02.620256 19106 external_mini_cluster.cc:1428] Setting key 18de177a9a02007790d474b5880cc59a
Aug 10 18:50:02 dist-test-slave-cbsf krb5kdc[31421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387802, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:50:02.644043 32238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:02.625260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:56165 (local address 127.18.168.190:35961)
0810 18:50:02.625397 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:02.625402 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:02.626614 (+ 1212us) server_negotiation.cc:407] Connection header received
0810 18:50:02.630252 (+ 3638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:02.630256 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:02.630314 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:02.630366 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:50:02.632516 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:02.633402 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:02.635426 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:02.635789 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:02.635865 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:02.639783 (+ 3918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:50:02.639786 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:50:02.639789 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:50:02.639817 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:50:02.641564 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:02.642171 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:02.642174 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:02.642176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:02.642281 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:02.642703 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:02.642706 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:02.642708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:02.642937 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:50:02.643012 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:02.643729 (+ 717us) server_negotiation.cc:299] Negotiation successful
0810 18:50:02.643872 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 18:50:02.646312 32559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:02.625757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:35961 (local address 127.18.168.131:56165)
0810 18:50:02.626463 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:02.626523 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 18:50:02.629976 (+ 3453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:50:02.630861 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:50:02.630870 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:50:02.631469 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:50:02.632343 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:02.632364 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:02.633804 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:02.633810 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:02.635289 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:02.635300 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:02.636066 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:02.636069 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:02.636317 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:02.637114 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:50:02.637135 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:50:02.639520 (+ 2385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:50:02.641707 (+ 2187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:02.641713 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:02.641729 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:02.642052 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:02.642389 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:02.642392 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:02.642394 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:02.642578 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:02.643101 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:50:02.643107 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:50:02.643426 (+ 319us) client_negotiation.cc:769] Sending connection context
0810 18:50:02.644486 (+ 1060us) client_negotiation.cc:240] Negotiation successful
0810 18:50:02.644916 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":225,"threads_started":1}
I20260810 18:50:02.647766 32557 heartbeater.cc:347] Connected to a master server at 127.18.168.190:35961
I20260810 18:50:02.648062 32557 heartbeater.cc:464] Registering TS with master...
I20260810 18:50:02.648614 32557 heartbeater.cc:511] Master 127.18.168.190:35961 requested a full tablet report, sending...
I20260810 18:50:02.649739 31984 ts_manager.cc:194] Registered new tserver with Master: a265e3ff08ab4dc8ac72a45b02156a5b (127.18.168.131:34709)
I20260810 18:50:02.650364 31984 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.131@KRBTEST.COM'} at 127.18.168.131:56165
I20260810 18:50:02.658829 19106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:50:02.673324 32238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:02.661789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46506 (local address 127.18.168.190:35961)
0810 18:50:02.661913 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:02.661919 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:02.662691 (+ 772us) server_negotiation.cc:407] Connection header received
0810 18:50:02.662900 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:02.662906 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:02.662981 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:02.663046 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:50:02.664189 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:02.665047 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:02.666361 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:02.666699 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:02.666789 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:02.669002 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:50:02.669007 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:50:02.669010 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:50:02.669040 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:50:02.671005 (+ 1965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:02.671616 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:02.671620 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:02.671622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:02.671733 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:02.672133 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:02.672138 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:02.672139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:02.672389 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:50:02.672522 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:02.672996 (+ 474us) server_negotiation.cc:299] Negotiation successful
0810 18:50:02.673137 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 18:50:02.677863 31984 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46506:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:50:02.682890 31984 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:50:02.708465 32572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:02.701744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:36171 (local address 127.0.0.1:38192)
0810 18:50:02.702309 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:02.702323 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:50:02.702464 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:50:02.703210 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:50:02.703216 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:50:02.703601 (+ 385us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:50:02.703711 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:02.703718 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:02.705340 (+ 1622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:02.705343 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:02.706979 (+ 1636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:02.706991 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:02.707836 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:02.707840 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:02.707953 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:02.707991 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 18:50:02.708119 (+ 128us) client_negotiation.cc:240] Negotiation successful
0810 18:50:02.708268 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"mutex_wait_us":153,"thread_start_us":82,"threads_started":1}
I20260810 18:50:02.709239 32573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:02.702164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38192 (local address 127.18.168.130:36171)
0810 18:50:02.702696 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:02.702703 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:02.702726 (+ 23us) server_negotiation.cc:407] Connection header received
0810 18:50:02.702807 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:02.702814 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:02.703023 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:02.703881 (+ 858us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:50:02.704054 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:02.705217 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:02.707117 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:02.707699 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:02.707838 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:02.707992 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:02.709001 (+ 1009us) server_negotiation.cc:299] Negotiation successful
0810 18:50:02.709102 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":87,"threads_started":1}
I20260810 18:50:02.710384 32571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:02.700808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42732 (local address 127.18.168.131:34709)
0810 18:50:02.701441 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:02.701467 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:02.701493 (+ 26us) server_negotiation.cc:407] Connection header received
0810 18:50:02.701577 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:02.701585 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:02.701801 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:02.702018 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:50:02.703894 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:02.705167 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:02.708640 (+ 3473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:02.709513 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:02.709609 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:02.709723 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:02.710097 (+ 374us) server_negotiation.cc:299] Negotiation successful
0810 18:50:02.710200 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":274,"threads_started":1}
I20260810 18:50:02.710899 32570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:02.699802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.131:34709 (local address 127.0.0.1:42732)
0810 18:50:02.700360 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:02.700406 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 18:50:02.700550 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:50:02.703489 (+ 2939us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:50:02.703493 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:50:02.703518 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:50:02.703710 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:02.703718 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:02.705340 (+ 1622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:02.705343 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:02.708442 (+ 3099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:02.708447 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:02.709822 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:02.709825 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:02.709941 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:02.709963 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 18:50:02.710702 (+ 739us) client_negotiation.cc:240] Negotiation successful
0810 18:50:02.710784 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"mutex_wait_us":95,"thread_start_us":199,"threads_started":1}
I20260810 18:50:02.712632 32306 tablet_service.cc:1467] Processing CreateTablet for tablet 92b6b0339461487eb8ec907920dfb734 (DEFAULT_TABLE table=test-table [id=31ad986fb8754bc5bf0bdd835f0d67c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:50:02.713830 32306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92b6b0339461487eb8ec907920dfb734. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:50:02.714895 32457 tablet_service.cc:1467] Processing CreateTablet for tablet 92b6b0339461487eb8ec907920dfb734 (DEFAULT_TABLE table=test-table [id=31ad986fb8754bc5bf0bdd835f0d67c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:50:02.715893 32457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92b6b0339461487eb8ec907920dfb734. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:50:02.727977 32576 tablet_bootstrap.cc:492] T 92b6b0339461487eb8ec907920dfb734 P c7e9fde0f7f142eebe48fbfa7585fdc0: Bootstrap starting.
I20260810 18:50:02.729269 32574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:02.702937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:40357 (local address 127.0.0.1:45796)
0810 18:50:02.708218 (+ 5281us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:02.708232 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:50:02.708359 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:50:02.711884 (+ 3525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:50:02.711888 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:50:02.711906 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:50:02.712083 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:02.712091 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:02.726081 (+ 13990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:02.726086 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:02.727289 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:02.727301 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:02.728687 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:02.728691 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:02.728816 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:02.728843 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 18:50:02.728922 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 18:50:02.729011 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5222,"thread_start_us":99,"threads_started":1}
I20260810 18:50:02.729913 32236 heartbeater.cc:503] Master 127.18.168.190:35961 was elected leader, sending a full tablet report...
I20260810 18:50:02.730558 32575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:02.703321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45796 (local address 127.18.168.129:40357)
0810 18:50:02.711257 (+ 7936us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:02.711264 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:02.711287 (+ 23us) server_negotiation.cc:407] Connection header received
0810 18:50:02.711357 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:02.711362 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:02.711540 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:02.711678 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:50:02.724857 (+ 13179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:02.725942 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:02.727456 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:02.728322 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:02.728458 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:02.728615 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:02.729738 (+ 1123us) server_negotiation.cc:299] Negotiation successful
0810 18:50:02.729922 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7830,"thread_start_us":135,"threads_started":1}
I20260810 18:50:02.731575 32576 tablet_bootstrap.cc:654] T 92b6b0339461487eb8ec907920dfb734 P c7e9fde0f7f142eebe48fbfa7585fdc0: Neither blocks nor log segments found. Creating new log.
I20260810 18:50:02.733108 32576 log.cc:824] T 92b6b0339461487eb8ec907920dfb734 P c7e9fde0f7f142eebe48fbfa7585fdc0: Log is configured to *not* fsync() on all Append() calls
I20260810 18:50:02.733680 32577 tablet_bootstrap.cc:492] T 92b6b0339461487eb8ec907920dfb734 P a265e3ff08ab4dc8ac72a45b02156a5b: Bootstrap starting.
I20260810 18:50:02.734768 32159 tablet_service.cc:1467] Processing CreateTablet for tablet 92b6b0339461487eb8ec907920dfb734 (DEFAULT_TABLE table=test-table [id=31ad986fb8754bc5bf0bdd835f0d67c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:50:02.736230 32159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92b6b0339461487eb8ec907920dfb734. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:50:02.737634 32577 tablet_bootstrap.cc:654] T 92b6b0339461487eb8ec907920dfb734 P a265e3ff08ab4dc8ac72a45b02156a5b: Neither blocks nor log segments found. Creating new log.
I20260810 18:50:02.739194 32577 log.cc:824] T 92b6b0339461487eb8ec907920dfb734 P a265e3ff08ab4dc8ac72a45b02156a5b: Log is configured to *not* fsync() on all Append() calls
I20260810 18:50:02.741984 32576 tablet_bootstrap.cc:492] T 92b6b0339461487eb8ec907920dfb734 P c7e9fde0f7f142eebe48fbfa7585fdc0: No bootstrap required, opened a new log
I20260810 18:50:02.742234 32576 ts_tablet_manager.cc:1397] T 92b6b0339461487eb8ec907920dfb734 P c7e9fde0f7f142eebe48fbfa7585fdc0: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.004s
I20260810 18:50:02.748507 32576 raft_consensus.cc:348] T 92b6b0339461487eb8ec907920dfb734 P c7e9fde0f7f142eebe48fbfa7585fdc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a265e3ff08ab4dc8ac72a45b02156a5b" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34709 } } peers { permanent_uuid: "c7e9fde0f7f142eebe48fbfa7585fdc0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36171 } } peers { permanent_uuid: "39aa5565bccb4e30a4c36b2b2cc98931" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 40357 } }
I20260810 18:50:02.748792 32576 raft_consensus.cc:374] T 92b6b0339461487eb8ec907920dfb734 P c7e9fde0f7f142eebe48fbfa7585fdc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:50:02.748867 32576 raft_consensus.cc:729] T 92b6b0339461487eb8ec907920dfb734 P c7e9fde0f7f142eebe48fbfa7585fdc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7e9fde0f7f142eebe48fbfa7585fdc0, State: Initialized, Role: FOLLOWER
I20260810 18:50:02.749678 32580 tablet_bootstrap.cc:492] T 92b6b0339461487eb8ec907920dfb734 P 39aa5565bccb4e30a4c36b2b2cc98931: Bootstrap starting.
I20260810 18:50:02.750694 32577 tablet_bootstrap.cc:492] T 92b6b0339461487eb8ec907920dfb734 P a265e3ff08ab4dc8ac72a45b02156a5b: No bootstrap required, opened a new log
I20260810 18:50:02.751117 32577 ts_tablet_manager.cc:1397] T 92b6b0339461487eb8ec907920dfb734 P a265e3ff08ab4dc8ac72a45b02156a5b: Time spent bootstrapping tablet: real 0.018s user 0.007s sys 0.000s
I20260810 18:50:02.752460 32580 tablet_bootstrap.cc:654] T 92b6b0339461487eb8ec907920dfb734 P 39aa5565bccb4e30a4c36b2b2cc98931: Neither blocks nor log segments found. Creating new log.
I20260810 18:50:02.753921 32580 log.cc:824] T 92b6b0339461487eb8ec907920dfb734 P 39aa5565bccb4e30a4c36b2b2cc98931: Log is configured to *not* fsync() on all Append() calls
I20260810 18:50:02.755823 32576 consensus_queue.cc:260] T 92b6b0339461487eb8ec907920dfb734 P c7e9fde0f7f142eebe48fbfa7585fdc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a265e3ff08ab4dc8ac72a45b02156a5b" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34709 } } peers { permanent_uuid: "c7e9fde0f7f142eebe48fbfa7585fdc0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36171 } } peers { permanent_uuid: "39aa5565bccb4e30a4c36b2b2cc98931" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 40357 } }
I20260810 18:50:02.757149 32577 raft_consensus.cc:348] T 92b6b0339461487eb8ec907920dfb734 P a265e3ff08ab4dc8ac72a45b02156a5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a265e3ff08ab4dc8ac72a45b02156a5b" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34709 } } peers { permanent_uuid: "c7e9fde0f7f142eebe48fbfa7585fdc0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36171 } } peers { permanent_uuid: "39aa5565bccb4e30a4c36b2b2cc98931" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 40357 } }
I20260810 18:50:02.757539 32577 raft_consensus.cc:374] T 92b6b0339461487eb8ec907920dfb734 P a265e3ff08ab4dc8ac72a45b02156a5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:50:02.757638 32577 raft_consensus.cc:729] T 92b6b0339461487eb8ec907920dfb734 P a265e3ff08ab4dc8ac72a45b02156a5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a265e3ff08ab4dc8ac72a45b02156a5b, State: Initialized, Role: FOLLOWER
I20260810 18:50:02.757681 32576 ts_tablet_manager.cc:1428] T 92b6b0339461487eb8ec907920dfb734 P c7e9fde0f7f142eebe48fbfa7585fdc0: Time spent starting tablet: real 0.015s user 0.004s sys 0.004s
I20260810 18:50:02.758072 32399 heartbeater.cc:503] Master 127.18.168.190:35961 was elected leader, sending a full tablet report...
I20260810 18:50:02.758181 32577 consensus_queue.cc:260] T 92b6b0339461487eb8ec907920dfb734 P a265e3ff08ab4dc8ac72a45b02156a5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a265e3ff08ab4dc8ac72a45b02156a5b" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34709 } } peers { permanent_uuid: "c7e9fde0f7f142eebe48fbfa7585fdc0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36171 } } peers { permanent_uuid: "39aa5565bccb4e30a4c36b2b2cc98931" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 40357 } }
I20260810 18:50:02.759246 32577 ts_tablet_manager.cc:1428] T 92b6b0339461487eb8ec907920dfb734 P a265e3ff08ab4dc8ac72a45b02156a5b: Time spent starting tablet: real 0.008s user 0.000s sys 0.008s
I20260810 18:50:02.759634 32557 heartbeater.cc:503] Master 127.18.168.190:35961 was elected leader, sending a full tablet report...
I20260810 18:50:02.761433 32580 tablet_bootstrap.cc:492] T 92b6b0339461487eb8ec907920dfb734 P 39aa5565bccb4e30a4c36b2b2cc98931: No bootstrap required, opened a new log
I20260810 18:50:02.761610 32580 ts_tablet_manager.cc:1397] T 92b6b0339461487eb8ec907920dfb734 P 39aa5565bccb4e30a4c36b2b2cc98931: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.000s
I20260810 18:50:02.768184 32580 raft_consensus.cc:348] T 92b6b0339461487eb8ec907920dfb734 P 39aa5565bccb4e30a4c36b2b2cc98931 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a265e3ff08ab4dc8ac72a45b02156a5b" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34709 } } peers { permanent_uuid: "c7e9fde0f7f142eebe48fbfa7585fdc0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36171 } } peers { permanent_uuid: "39aa5565bccb4e30a4c36b2b2cc98931" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 40357 } }
I20260810 18:50:02.768460 32580 raft_consensus.cc:374] T 92b6b0339461487eb8ec907920dfb734 P 39aa5565bccb4e30a4c36b2b2cc98931 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:50:02.768507 32580 raft_consensus.cc:729] T 92b6b0339461487eb8ec907920dfb734 P 39aa5565bccb4e30a4c36b2b2cc98931 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39aa5565bccb4e30a4c36b2b2cc98931, State: Initialized, Role: FOLLOWER
I20260810 18:50:02.776597 32580 consensus_queue.cc:260] T 92b6b0339461487eb8ec907920dfb734 P 39aa5565bccb4e30a4c36b2b2cc98931 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a265e3ff08ab4dc8ac72a45b02156a5b" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34709 } } peers { permanent_uuid: "c7e9fde0f7f142eebe48fbfa7585fdc0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36171 } } peers { permanent_uuid: "39aa5565bccb4e30a4c36b2b2cc98931" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 40357 } }
I20260810 18:50:02.778254 32580 ts_tablet_manager.cc:1428] T 92b6b0339461487eb8ec907920dfb734 P 39aa5565bccb4e30a4c36b2b2cc98931: Time spent starting tablet: real 0.017s user 0.005s sys 0.003s
W20260810 18:50:02.874929 32558 tablet.cc:2369] T 92b6b0339461487eb8ec907920dfb734 P a265e3ff08ab4dc8ac72a45b02156a5b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:50:02.908541 32400 tablet.cc:2369] T 92b6b0339461487eb8ec907920dfb734 P c7e9fde0f7f142eebe48fbfa7585fdc0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:50:02.949890 32237 tablet.cc:2369] T 92b6b0339461487eb8ec907920dfb734 P 39aa5565bccb4e30a4c36b2b2cc98931: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:50:02.973129 32584 raft_consensus.cc:482] T 92b6b0339461487eb8ec907920dfb734 P a265e3ff08ab4dc8ac72a45b02156a5b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:50:02.973412 32584 raft_consensus.cc:504] T 92b6b0339461487eb8ec907920dfb734 P a265e3ff08ab4dc8ac72a45b02156a5b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a265e3ff08ab4dc8ac72a45b02156a5b" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34709 } } peers { permanent_uuid: "c7e9fde0f7f142eebe48fbfa7585fdc0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36171 } } peers { permanent_uuid: "39aa5565bccb4e30a4c36b2b2cc98931" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 40357 } }
I20260810 18:50:02.974911 32584 leader_election.cc:288] T 92b6b0339461487eb8ec907920dfb734 P a265e3ff08ab4dc8ac72a45b02156a5b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7e9fde0f7f142eebe48fbfa7585fdc0 (127.18.168.130:36171), 39aa5565bccb4e30a4c36b2b2cc98931 (127.18.168.129:40357)
Aug 10 18:50:02 dist-test-slave-cbsf krb5kdc[31421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387801, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for HTTP/127.18.168.148@KRBTEST.COM
I20260810 18:50:02.983299 32559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:02.975206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:36171 (local address 127.18.168.131:38383)
0810 18:50:02.976397 (+ 1191us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:02.976415 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:50:02.976524 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:50:02.977839 (+ 1315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:50:02.977843 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:50:02.977865 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:50:02.978097 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:02.978106 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:02.980157 (+ 2051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:02.980161 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:02.981526 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:02.981538 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:02.982721 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:02.982725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:02.982936 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:02.982967 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 18:50:02.983054 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 18:50:02.983140 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1086}
I20260810 18:50:02.985023 32573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:02.976892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:38383 (local address 127.18.168.130:36171)
0810 18:50:02.977233 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:02.977238 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:02.977257 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:50:02.977322 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:02.977326 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:02.977377 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:02.977463 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:50:02.978259 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:02.979085 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:02.981692 (+ 2607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:02.982434 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:02.982567 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:02.982650 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:02.984775 (+ 2125us) server_negotiation.cc:299] Negotiation successful
0810 18:50:02.984873 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237}
I20260810 18:50:02.986023 32344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92b6b0339461487eb8ec907920dfb734" candidate_uuid: "a265e3ff08ab4dc8ac72a45b02156a5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7e9fde0f7f142eebe48fbfa7585fdc0" is_pre_election: true
I20260810 18:50:02.986423 32344 raft_consensus.cc:2454] T 92b6b0339461487eb8ec907920dfb734 P c7e9fde0f7f142eebe48fbfa7585fdc0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a265e3ff08ab4dc8ac72a45b02156a5b in term 0.
I20260810 18:50:02.987335 32423 leader_election.cc:302] T 92b6b0339461487eb8ec907920dfb734 P a265e3ff08ab4dc8ac72a45b02156a5b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a265e3ff08ab4dc8ac72a45b02156a5b, c7e9fde0f7f142eebe48fbfa7585fdc0; no voters:
I20260810 18:50:02.987706 32584 raft_consensus.cc:2781] T 92b6b0339461487eb8ec907920dfb734 P a265e3ff08ab4dc8ac72a45b02156a5b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:50:02.987819 32584 raft_consensus.cc:482] T 92b6b0339461487eb8ec907920dfb734 P a265e3ff08ab4dc8ac72a45b02156a5b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:50:02.987866 32584 raft_consensus.cc:3037] T 92b6b0339461487eb8ec907920dfb734 P a265e3ff08ab4dc8ac72a45b02156a5b [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:50:02.989367 32584 raft_consensus.cc:504] T 92b6b0339461487eb8ec907920dfb734 P a265e3ff08ab4dc8ac72a45b02156a5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a265e3ff08ab4dc8ac72a45b02156a5b" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34709 } } peers { permanent_uuid: "c7e9fde0f7f142eebe48fbfa7585fdc0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36171 } } peers { permanent_uuid: "39aa5565bccb4e30a4c36b2b2cc98931" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 40357 } }
I20260810 18:50:02.989765 32584 leader_election.cc:288] T 92b6b0339461487eb8ec907920dfb734 P a265e3ff08ab4dc8ac72a45b02156a5b [CANDIDATE]: Term 1 election: Requested vote from peers c7e9fde0f7f142eebe48fbfa7585fdc0 (127.18.168.130:36171), 39aa5565bccb4e30a4c36b2b2cc98931 (127.18.168.129:40357)
I20260810 18:50:02.990442 32344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92b6b0339461487eb8ec907920dfb734" candidate_uuid: "a265e3ff08ab4dc8ac72a45b02156a5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7e9fde0f7f142eebe48fbfa7585fdc0"
I20260810 18:50:02.990579 32344 raft_consensus.cc:3037] T 92b6b0339461487eb8ec907920dfb734 P c7e9fde0f7f142eebe48fbfa7585fdc0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:50:03.003533 32344 raft_consensus.cc:2454] T 92b6b0339461487eb8ec907920dfb734 P c7e9fde0f7f142eebe48fbfa7585fdc0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a265e3ff08ab4dc8ac72a45b02156a5b in term 1.
I20260810 18:50:03.006474 32423 leader_election.cc:302] T 92b6b0339461487eb8ec907920dfb734 P a265e3ff08ab4dc8ac72a45b02156a5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a265e3ff08ab4dc8ac72a45b02156a5b, c7e9fde0f7f142eebe48fbfa7585fdc0; no voters:
I20260810 18:50:03.006681 32584 raft_consensus.cc:2781] T 92b6b0339461487eb8ec907920dfb734 P a265e3ff08ab4dc8ac72a45b02156a5b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:50:03.006982 32584 raft_consensus.cc:686] T 92b6b0339461487eb8ec907920dfb734 P a265e3ff08ab4dc8ac72a45b02156a5b [term 1 LEADER]: Becoming Leader. State: Replica: a265e3ff08ab4dc8ac72a45b02156a5b, State: Running, Role: LEADER
I20260810 18:50:03.007386 32584 consensus_queue.cc:237] T 92b6b0339461487eb8ec907920dfb734 P a265e3ff08ab4dc8ac72a45b02156a5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a265e3ff08ab4dc8ac72a45b02156a5b" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34709 } } peers { permanent_uuid: "c7e9fde0f7f142eebe48fbfa7585fdc0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36171 } } peers { permanent_uuid: "39aa5565bccb4e30a4c36b2b2cc98931" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 40357 } }
I20260810 18:50:03.011551 31984 catalog_manager.cc:5495] T 92b6b0339461487eb8ec907920dfb734 P a265e3ff08ab4dc8ac72a45b02156a5b reported cstate change: term changed from 0 to 1, leader changed from <none> to a265e3ff08ab4dc8ac72a45b02156a5b (127.18.168.131). New cstate: current_term: 1 leader_uuid: "a265e3ff08ab4dc8ac72a45b02156a5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a265e3ff08ab4dc8ac72a45b02156a5b" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 34709 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7e9fde0f7f142eebe48fbfa7585fdc0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39aa5565bccb4e30a4c36b2b2cc98931" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 40357 } health_report { overall_health: UNKNOWN } } }
I20260810 18:50:03.014278 32559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:02.975623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:40357 (local address 127.18.168.131:59359)
0810 18:50:02.983476 (+ 7853us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:02.983494 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:50:02.988305 (+ 4811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:50:03.004187 (+ 15882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:50:03.004190 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:50:03.004215 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:50:03.004317 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:03.004323 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:03.005444 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:03.005447 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:03.009815 (+ 4368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:03.009828 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:03.013718 (+ 3890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:03.013722 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:03.013819 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:03.013842 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 18:50:03.013912 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 18:50:03.013991 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7795,"thread_start_us":360,"threads_started":1}
I20260810 18:50:03.014940 32575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:02.977071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:59359 (local address 127.18.168.129:40357)
0810 18:50:02.977652 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:02.977657 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:02.983564 (+ 5907us) server_negotiation.cc:407] Connection header received
0810 18:50:03.003897 (+ 20333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:03.003901 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:03.003965 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:03.004022 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:50:03.004480 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:03.005281 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:03.012795 (+ 7514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:03.013471 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:03.013597 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:03.013655 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:03.014730 (+ 1075us) server_negotiation.cc:299] Negotiation successful
0810 18:50:03.014824 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490}
I20260810 18:50:03.015851 32182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92b6b0339461487eb8ec907920dfb734" candidate_uuid: "a265e3ff08ab4dc8ac72a45b02156a5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39aa5565bccb4e30a4c36b2b2cc98931" is_pre_election: true
I20260810 18:50:03.016319 32182 raft_consensus.cc:2454] T 92b6b0339461487eb8ec907920dfb734 P 39aa5565bccb4e30a4c36b2b2cc98931 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a265e3ff08ab4dc8ac72a45b02156a5b in term 0.
I20260810 18:50:03.015849 32181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92b6b0339461487eb8ec907920dfb734" candidate_uuid: "a265e3ff08ab4dc8ac72a45b02156a5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39aa5565bccb4e30a4c36b2b2cc98931"
I20260810 18:50:03.016723 32181 raft_consensus.cc:3037] T 92b6b0339461487eb8ec907920dfb734 P 39aa5565bccb4e30a4c36b2b2cc98931 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:50:03.018808 32181 raft_consensus.cc:2454] T 92b6b0339461487eb8ec907920dfb734 P 39aa5565bccb4e30a4c36b2b2cc98931 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a265e3ff08ab4dc8ac72a45b02156a5b in term 1.
I20260810 18:50:03.083921 32571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:03.078205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42746 (local address 127.18.168.131:34709)
0810 18:50:03.078329 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:03.078334 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:03.078385 (+ 51us) server_negotiation.cc:407] Connection header received
0810 18:50:03.078605 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:03.078611 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:03.078708 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:03.078821 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:50:03.079412 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:03.080162 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:03.081437 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:03.081760 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:03.081861 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:03.082292 (+ 431us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:50:03.082901 (+ 609us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:50:03.083048 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:03.083553 (+ 505us) server_negotiation.cc:299] Negotiation successful
0810 18:50:03.083624 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 18:50:03.096323 32181 raft_consensus.cc:1261] T 92b6b0339461487eb8ec907920dfb734 P 39aa5565bccb4e30a4c36b2b2cc98931 [term 1 FOLLOWER]: Refusing update from remote peer a265e3ff08ab4dc8ac72a45b02156a5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:50:03.097271 32588 consensus_queue.cc:1035] T 92b6b0339461487eb8ec907920dfb734 P a265e3ff08ab4dc8ac72a45b02156a5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "39aa5565bccb4e30a4c36b2b2cc98931" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:50:03.104555 32344 raft_consensus.cc:1261] T 92b6b0339461487eb8ec907920dfb734 P c7e9fde0f7f142eebe48fbfa7585fdc0 [term 1 FOLLOWER]: Refusing update from remote peer a265e3ff08ab4dc8ac72a45b02156a5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:50:03.105334 32588 consensus_queue.cc:1035] T 92b6b0339461487eb8ec907920dfb734 P a265e3ff08ab4dc8ac72a45b02156a5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7e9fde0f7f142eebe48fbfa7585fdc0" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:50:03.123561 32593 mvcc.cc:205] Tried to move back new op lower bound from 7317044441472438272 to 7317044441121554432. Current Snapshot: MvccSnapshot[applied={T|T < 7317044441472438272}]
I20260810 18:50:03.131918 32594 mvcc.cc:205] Tried to move back new op lower bound from 7317044441472438272 to 7317044441121554432. Current Snapshot: MvccSnapshot[applied={T|T < 7317044441472438272}]
I20260810 18:50:03.132201 32596 mvcc.cc:205] Tried to move back new op lower bound from 7317044441472438272 to 7317044441121554432. Current Snapshot: MvccSnapshot[applied={T|T < 7317044441472438272}]
I20260810 18:50:03.144639 31984 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46506:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:50:03.144917 31984 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46506:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:50:03.148185 31984 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ae543986c9694a3e9e5bbc0580a74307: Sending DeleteTablet for 3 replicas of tablet 92b6b0339461487eb8ec907920dfb734
I20260810 18:50:03.149822 32306 tablet_service.cc:1514] Processing DeleteTablet for tablet 92b6b0339461487eb8ec907920dfb734 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:50:03 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:38192
I20260810 18:50:03.150839 32159 tablet_service.cc:1514] Processing DeleteTablet for tablet 92b6b0339461487eb8ec907920dfb734 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:50:03 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:45796
I20260810 18:50:03.152324 32457 tablet_service.cc:1514] Processing DeleteTablet for tablet 92b6b0339461487eb8ec907920dfb734 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:50:03 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:42732
I20260810 18:50:03.153666 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 32082
I20260810 18:50:03.157189 32602 tablet_replica.cc:330] T 92b6b0339461487eb8ec907920dfb734 P c7e9fde0f7f142eebe48fbfa7585fdc0: stopping tablet replica
I20260810 18:50:03.157584 32602 raft_consensus.cc:2229] T 92b6b0339461487eb8ec907920dfb734 P c7e9fde0f7f142eebe48fbfa7585fdc0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:50:03.157945 32602 raft_consensus.cc:2258] T 92b6b0339461487eb8ec907920dfb734 P c7e9fde0f7f142eebe48fbfa7585fdc0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:50:03.164961 32604 tablet_replica.cc:330] T 92b6b0339461487eb8ec907920dfb734 P a265e3ff08ab4dc8ac72a45b02156a5b: stopping tablet replica
I20260810 18:50:03.165308 32604 raft_consensus.cc:2229] T 92b6b0339461487eb8ec907920dfb734 P a265e3ff08ab4dc8ac72a45b02156a5b [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:50:03.165772 32604 raft_consensus.cc:2258] T 92b6b0339461487eb8ec907920dfb734 P a265e3ff08ab4dc8ac72a45b02156a5b [term 1 FOLLOWER]: Raft consensus is shut down!
W20260810 18:50:03.167450 31968 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.168.129:40357 (error 108)
I20260810 18:50:03.168437 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 32243
I20260810 18:50:03.171562 32604 ts_tablet_manager.cc:1905] T 92b6b0339461487eb8ec907920dfb734 P a265e3ff08ab4dc8ac72a45b02156a5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:50:03.174700 32604 ts_tablet_manager.cc:1918] T 92b6b0339461487eb8ec907920dfb734 P a265e3ff08ab4dc8ac72a45b02156a5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:50:03.174813 32604 log.cc:1196] T 92b6b0339461487eb8ec907920dfb734 P a265e3ff08ab4dc8ac72a45b02156a5b: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387687857264-19106-0/minicluster-data/ts-2/wal/wals/92b6b0339461487eb8ec907920dfb734
I20260810 18:50:03.175056 32574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:03.174644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:40357 (local address 127.0.0.1:45804)
0810 18:50:03.174863 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:03.174952 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.129:40357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
I20260810 18:50:03.175189 32604 ts_tablet_manager.cc:1939] T 92b6b0339461487eb8ec907920dfb734 P a265e3ff08ab4dc8ac72a45b02156a5b: Deleting consensus metadata
W20260810 18:50:03.175519 31968 catalog_manager.cc:4584] TS 39aa5565bccb4e30a4c36b2b2cc98931 (127.18.168.129:40357): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 92b6b0339461487eb8ec907920dfb734: Network error: Client connection negotiation failed: client connection to 127.18.168.129:40357: connect: Connection refused (error 111)
I20260810 18:50:03.176555 31969 catalog_manager.cc:4841] TS a265e3ff08ab4dc8ac72a45b02156a5b (127.18.168.131:34709): tablet 92b6b0339461487eb8ec907920dfb734 (table test-table [id=31ad986fb8754bc5bf0bdd835f0d67c2]) successfully deleted
I20260810 18:50:03.184063 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 32404
I20260810 18:50:03.184252 32574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:03.183915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:36171 (local address 127.0.0.1:38204)
0810 18:50:03.184107 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:03.184170 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.130:36171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260810 18:50:03.202003 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 31946
I20260810 18:50:03.214702 19106 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 18:50:03.559263 19106 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 18:50:03.559381 19106 mini_ranger.cc:66] Stopping Ranger...
I20260810 18:50:03.904325 19106 mini_ranger.cc:68] Stopped Ranger
2026-08-10 18:50:03.904 UTC [31460] LOG: received smart shutdown request
2026-08-10 18:50:03.909 UTC [31460] LOG: background worker "logical replication launcher" (PID 31469) exited with exit code 1
2026-08-10 18:50:03.909 UTC [31464] LOG: shutting down
2026-08-10 18:50:04.085 UTC [31460] LOG: database system is shut down
2026-08-10T18:50:04Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50764 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:50:04 dist-test-slave-cbsf krb5kdc[32616](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:50:04 dist-test-slave-cbsf krb5kdc[32616](info): set up 2 sockets
Aug 10 18:50:04 dist-test-slave-cbsf krb5kdc[32616](info): commencing operation
krb5kdc: starting...
W20260810 18:50:05.367208 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:50:05 dist-test-slave-cbsf krb5kdc[32616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:50:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:50:05Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.18.168.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.18.168.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.18.168.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/rangeradmin_127.18.168.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.18.168.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/rangeradmin_127.18.168.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.18.168.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.18.168.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.18.168.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/rangerlookup_127.18.168.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.18.168.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/rangerlookup_127.18.168.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.168.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.18.168.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.168.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/HTTP_127.18.168.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.168.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/HTTP_127.18.168.148@KRBTEST.COM.keytab.
I20260810 18:50:05.543062 19106 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task1VMCzb/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/postgres -l logfile start
2026-08-10 18:50:08.009 UTC [32654] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 18:50:08.009 UTC [32654] LOG: listening on IPv4 address "127.18.168.148", port 57013
2026-08-10 18:50:08.013 UTC [32654] LOG: listening on Unix socket "/tmp/.s.PGSQL.57013"
2026-08-10 18:50:08.035 UTC [32658] LOG: database system was shut down at 2026-08-10 18:50:06 UTC
2026-08-10 18:50:08.039 UTC [32654] LOG: database system is ready to accept connections
I20260810 18:50:09.050616 19106 mini_postgres.cc:96] Postgres bound to 57013
2026-08-10 18:50:09.058 UTC [32666] FATAL: database "slave" does not exist
127.18.168.148:57013 - accepting connections
I20260810 18:50:09.058723 19106 mini_ranger.cc:161] Starting Ranger...
I20260810 18:50:09.074294 19106 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 18:50:09.557016 19106 mini_ranger.cc:87] Created ranger Postgres database
I20260810 18:50:09.557116 19106 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ranger-admin
2026-08-10 18:50:09,844 [I] DB FLAVOR :POSTGRES
2026-08-10 18:50:09,844 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 18:50:09,844 [I] Checking connection..
2026-08-10 18:50:09,844 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:57013/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 18:50:10,122 [I] Checking connection passed.
2026-08-10 18:50:10,122 [I] --------- Verifying version history table ---------
2026-08-10 18:50:10,123 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:57013/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 18:50:10,367 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 18:50:10,368 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:57013/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 18:50:10,622 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 18:50:10,622 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 18:50:10,622 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:57013/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 18:50:10.890 UTC [32738] WARNING: there is no transaction in progress
2026-08-10 18:50:10,899 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 18:50:10,899 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:57013/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 18:50:11,168 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 18:50:11,169 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 18:50:11,169 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:57013/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 18:50:11,416 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:57013/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 18:50:11,680 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:57013/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-cbsf.c.gcp-upstream.internal','N') ;"
2026-08-10 18:50:11,932 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:57013/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 18:50:12,191 [I] Table x_portal_user does not exist in database ranger
2026-08-10 18:50:12,192 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:57013/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 18:50:12,475 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 18:50:12,476 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:57013/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 18:50:12,709 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:57013/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 18:50:12,972 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 18:50:12,973 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:57013/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 18:50:13.938 UTC [400] WARNING: there is no transaction in progress
2026-08-10 18:50:13.955 UTC [400] WARNING: there is no transaction in progress
2026-08-10 18:50:13.971 UTC [400] WARNING: there is no transaction in progress
2026-08-10 18:50:13.987 UTC [400] WARNING: there is no transaction in progress
2026-08-10 18:50:14.005 UTC [400] WARNING: there is no transaction in progress
2026-08-10 18:50:14.026 UTC [400] WARNING: there is no transaction in progress
2026-08-10 18:50:14.090 UTC [400] WARNING: there is no transaction in progress
2026-08-10 18:50:14.097 UTC [400] WARNING: there is no transaction in progress
2026-08-10 18:50:14.111 UTC [400] WARNING: there is no transaction in progress
2026-08-10 18:50:14.121 UTC [400] WARNING: there is no transaction in progress
2026-08-10 18:50:14.131 UTC [400] WARNING: there is no transaction in progress
2026-08-10 18:50:14.145 UTC [400] WARNING: there is no transaction in progress
2026-08-10 18:50:14.154 UTC [400] WARNING: there is no transaction in progress
2026-08-10 18:50:14.160 UTC [400] WARNING: there is no transaction in progress
2026-08-10 18:50:14.166 UTC [400] WARNING: there is no transaction in progress
2026-08-10 18:50:14.173 UTC [400] WARNING: there is no transaction in progress
2026-08-10 18:50:14,788 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 18:50:14,788 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:57013/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 18:50:15,050 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 18:50:15,050 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:57013/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 18:50:15,312 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 18:50:15,312 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:57013/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 18:50:15,582 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 18:50:15,582 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:57013/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 18:50:15,844 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:57013/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 18:50:16,117 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:57013/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-cbsf.c.gcp-upstream.internal';"
2026-08-10 18:50:16,366 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 18:50:16,367 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:57013/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 18:50:16,627 [I] DB_PATCHES have already been applied
I20260810 18:50:16.633055 19106 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ranger-admin:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task1VMCzb/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 18:50:16.633143 19106 mini_ranger.cc:193] Using host: 127.18.168.148
I20260810 18:50:16.637557 19106 mini_ranger.cc:227] Ranger admin URL: http://127.18.168.148:46601
Aug 10, 2026 6:50:16 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 6:50:16 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 6:50:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 6:50:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 6:50:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 6:50:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 6:50:17 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 18:50:17.865567 19106 mini_ranger.cc:160] Time spent starting Ranger: real 8.807s user 0.000s sys 0.011s
Aug 10, 2026 6:50:37 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 6:50:37 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 10, 2026 6:50:37 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 6:50:37 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 6:50:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 6:50:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 6:50:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 6:50:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 6:50:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 6:50:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 6:50:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 6:50:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 6:50:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 6:50:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 6:50:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 6:50:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 6:50:37 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 6:50:38 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 18:50:38.596313 19106 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.18.168.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.18.168.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.18.168.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/rangerkms_127.18.168.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.18.168.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/rangerkms_127.18.168.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.168.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.18.168.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.168.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/HTTP_127.18.168.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.168.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/HTTP_127.18.168.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 18:50:38 dist-test-slave-cbsf krb5kdc[32616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387838, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 18:50:38.712013 19106 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 18:50:38.726980 19106 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 18:50:39.617024 19106 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 18:50:39.617120 19106 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ranger-kms
I20260810 18:50:39.617141 19106 mini_ranger_kms.cc:227] Using postgres at 127.18.168.148:57013
2026-08-10 18:50:39,948 [I] DB FLAVOR :POSTGRES
2026-08-10 18:50:39,949 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 18:50:39,949 [I] Checking connection
2026-08-10 18:50:39,949 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:57013/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 18:50:40,241 [I] connection success
2026-08-10 18:50:40,241 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 18:50:40,241 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 18:50:40,241 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:57013/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 18:50:40,494 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 18:50:40,494 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 18:50:40,494 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 18:50:40,495 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.168.148:57013/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 18:50:40,769 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 18:50:41.236810 19106 mini_ranger_kms.cc:310] Created kms service
I20260810 18:50:41.416541 19106 mini_ranger_kms.cc:326] Created kudu user
I20260810 18:50:41.460479 19106 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 18:50:41 dist-test-slave-cbsf krb5kdc[32616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387838, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.18.168.148@KRBTEST.COM
I20260810 18:50:41.887048 19106 mini_ranger_kms.cc:384] Added ranger policy
I20260810 18:50:41.887207 19106 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ranger-kms:/tmp/dist-test-task1VMCzb/build/debug/bin/postgresql.jar:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task1VMCzb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 18:50:41.887231 19106 mini_ranger_kms.cc:240] Using host: 127.18.168.148
I20260810 18:50:41.892585 19106 mini_ranger_kms.cc:276] Ranger KMS PID: 602
I20260810 18:50:41.892676 19106 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.18.168.148:48549
Aug 10, 2026 6:50:42 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 6:50:42 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 6:50:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 6:50:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 6:50:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 6:50:43 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 10 18:50:44 dist-test-slave-cbsf krb5kdc[32616](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786387844, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.168.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:50:44.391610 19106 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.680s user 0.011s sys 0.293s
I20260810 18:50:44.391752 19106 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 18:50:44.391848 19106 mini_ranger_kms.cc:401] 127.18.168.148:48549/kms/v1/keys
Aug 10 18:50:44 dist-test-slave-cbsf krb5kdc[32616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387844, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.168.148@KRBTEST.COM for HTTP/127.18.168.148@KRBTEST.COM
Aug 10 18:50:45 dist-test-slave-cbsf krb5kdc[32616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387844, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.168.148@KRBTEST.COM for HTTP/127.18.168.148@KRBTEST.COM
Aug 10 18:50:47 dist-test-slave-cbsf krb5kdc[32616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.18.168.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
I20260810 18:50:47.363809 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:43801
--webserver_interface=127.18.168.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:38753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:43801
--ranger_config_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.168.148:48549/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:50:47.475992 661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:50:47.476238 661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:50:47.476275 661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:50:47.480430 661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:50:47.480485 661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:50:47.480505 661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:50:47.480553 661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:50:47.480587 661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:50:47.485962 661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:38753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/master-0/wal
--ranger_kms_url=127.18.168.148:48549/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:43801
--ranger_config_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:43801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:50:47.487094 661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:50:47.488154 661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:50:47.496721 668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:50:47.496872 661 server_base.cc:1034] running on GCE node
W20260810 18:50:47.496721 667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:50:47.496738 670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:50:47.497622 661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:50:47.498673 661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:50:47.499837 661 hybrid_clock.cc:648] HybridClock initialized: now 1786387847499819 us; error 30 us; skew 500 ppm
Aug 10 18:50:47 dist-test-slave-cbsf krb5kdc[32616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387847, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:50:47.506556 661 init.cc:377] Logged in from keytab as kudu/127.18.168.190@KRBTEST.COM (short username kudu)
I20260810 18:50:47.507460 661 webserver.cc:460] Webserver started at http://127.18.168.190:38075/ using document root <none> and password file <none>
I20260810 18:50:47.508008 661 fs_manager.cc:359] Metadata directory not provided
I20260810 18:50:47.508092 661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:50:47.508313 661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 18:50:47 dist-test-slave-cbsf krb5kdc[32616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387847, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for HTTP/127.18.168.148@KRBTEST.COM
I20260810 18:50:47.566733 661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "db5a963396c94f9ca5b846feadc654f7"
format_stamp: "Formatted at 2026-08-10 18:50:47 on dist-test-slave-cbsf"
server_key: "96281fc55bea126fbcd2e3a59ae4082a"
server_key_iv: "128e051848ee13d0aff34a455e64e65c"
server_key_version: "kuduclusterkey@0"
I20260810 18:50:47.567390 661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "db5a963396c94f9ca5b846feadc654f7"
format_stamp: "Formatted at 2026-08-10 18:50:47 on dist-test-slave-cbsf"
server_key: "96281fc55bea126fbcd2e3a59ae4082a"
server_key_iv: "128e051848ee13d0aff34a455e64e65c"
server_key_version: "kuduclusterkey@0"
I20260810 18:50:47.571333 661 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:50:47.584882 677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:47.586285 661 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 18:50:47.586434 661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "db5a963396c94f9ca5b846feadc654f7"
format_stamp: "Formatted at 2026-08-10 18:50:47 on dist-test-slave-cbsf"
server_key: "96281fc55bea126fbcd2e3a59ae4082a"
server_key_iv: "128e051848ee13d0aff34a455e64e65c"
server_key_version: "kuduclusterkey@0"
I20260810 18:50:47.586571 661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:50:47.599924 661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:50:47.602850 661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:50:47.603058 661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:50:47.611236 661 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:43801
I20260810 18:50:47.611302 736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:43801 every 8 connection(s)
I20260810 18:50:47.612227 661 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:50:47.615620 737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:50:47.621215 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 661
I20260810 18:50:47.621347 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:50:47.621677 19106 external_mini_cluster.cc:1428] Setting key bc0235ef71c0384596f8c98fb0ce2200
I20260810 18:50:47.622815 737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7: Bootstrap starting.
I20260810 18:50:47.625667 737 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7: Neither blocks nor log segments found. Creating new log.
I20260810 18:50:47.626535 737 log.cc:824] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7: Log is configured to *not* fsync() on all Append() calls
I20260810 18:50:47.628881 737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7: No bootstrap required, opened a new log
Aug 10 18:50:47 dist-test-slave-cbsf krb5kdc[32616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:50:47.633150 737 raft_consensus.cc:348] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5a963396c94f9ca5b846feadc654f7" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43801 } }
I20260810 18:50:47.633402 737 raft_consensus.cc:374] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:50:47.633456 737 raft_consensus.cc:729] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db5a963396c94f9ca5b846feadc654f7, State: Initialized, Role: FOLLOWER
I20260810 18:50:47.633936 737 consensus_queue.cc:260] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5a963396c94f9ca5b846feadc654f7" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43801 } }
I20260810 18:50:47.634071 737 raft_consensus.cc:388] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:50:47.634135 737 raft_consensus.cc:482] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:50:47.634215 737 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:50:47.635458 737 raft_consensus.cc:504] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5a963396c94f9ca5b846feadc654f7" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43801 } }
I20260810 18:50:47.635802 737 leader_election.cc:302] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db5a963396c94f9ca5b846feadc654f7; no voters:
I20260810 18:50:47.636165 737 leader_election.cc:288] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:50:47.636273 743 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:50:47.636379 740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:47.623175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39882 (local address 127.18.168.190:43801)
0810 18:50:47.623615 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:47.623634 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:47.623670 (+ 36us) server_negotiation.cc:407] Connection header received
0810 18:50:47.624284 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:47.624299 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:47.624626 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:47.625020 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:50:47.625899 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:47.627098 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:47.628277 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:47.628847 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:47.629037 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:47.631532 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:50:47.631539 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:50:47.631557 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:50:47.631592 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:50:47.633352 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:47.633836 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:47.633842 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:47.633848 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:47.633959 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:47.634284 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:47.634287 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:47.634289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:47.634698 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:50:47.634902 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:47.635242 (+ 340us) server_negotiation.cc:299] Negotiation successful
0810 18:50:47.635545 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":144,"threads_started":1}
I20260810 18:50:47.636473 743 raft_consensus.cc:686] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7 [term 1 LEADER]: Becoming Leader. State: Replica: db5a963396c94f9ca5b846feadc654f7, State: Running, Role: LEADER
I20260810 18:50:47.636845 743 consensus_queue.cc:237] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5a963396c94f9ca5b846feadc654f7" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43801 } }
I20260810 18:50:47.638082 743 sys_catalog.cc:455] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db5a963396c94f9ca5b846feadc654f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5a963396c94f9ca5b846feadc654f7" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43801 } } }
I20260810 18:50:47.638293 743 sys_catalog.cc:458] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7 [sys.catalog]: This master's current role is: LEADER
I20260810 18:50:47.638574 743 sys_catalog.cc:455] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db5a963396c94f9ca5b846feadc654f7. Latest consensus state: current_term: 1 leader_uuid: "db5a963396c94f9ca5b846feadc654f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5a963396c94f9ca5b846feadc654f7" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43801 } } }
I20260810 18:50:47.638731 743 sys_catalog.cc:458] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7 [sys.catalog]: This master's current role is: LEADER
I20260810 18:50:47.639655 737 sys_catalog.cc:564] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:50:47.641965 737 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 18:50:47.645864 748 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:50:47.650192 748 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:50:47.657137 748 catalog_manager.cc:1347] Generated new cluster ID: f0f42adc87fe4d91b3f7a4dbae6fefe0
I20260810 18:50:47.657223 748 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:50:47.664613 748 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:50:47.665946 748 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:50:47.681123 748 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7: Generated new TSK 0
I20260810 18:50:47.682065 748 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.168.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab.
Entry for principal kudu/127.18.168.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab.
I20260810 18:50:48.413987 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.168.148:48549/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.129:0
--local_ip_for_outbound_sockets=127.18.168.129
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:43801
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:38753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:50:48.572610 791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:50:48.572939 791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:50:48.573047 791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:50:48.577286 791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:50:48.577355 791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:50:48.577438 791 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.129
I20260810 18:50:48.582456 791 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:38753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.18.168.148:48549/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:43801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.791
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.129
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:50:48.583462 791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:50:48.584492 791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:50:48.595968 797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:50:48.596534 796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:50:48.597813 791 server_base.cc:1034] running on GCE node
W20260810 18:50:48.598227 799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:50:48.598886 791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:50:48.599764 791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:50:48.607203 791 hybrid_clock.cc:648] HybridClock initialized: now 1786387848607144 us; error 67 us; skew 500 ppm
Aug 10 18:50:48 dist-test-slave-cbsf krb5kdc[32616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387848, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:50:48.611131 791 init.cc:377] Logged in from keytab as kudu/127.18.168.129@KRBTEST.COM (short username kudu)
I20260810 18:50:48.612403 791 webserver.cc:460] Webserver started at http://127.18.168.129:45999/ using document root <none> and password file <none>
I20260810 18:50:48.613334 791 fs_manager.cc:359] Metadata directory not provided
I20260810 18:50:48.613508 791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:50:48.613855 791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 18:50:48 dist-test-slave-cbsf krb5kdc[32616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387848, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for HTTP/127.18.168.148@KRBTEST.COM
I20260810 18:50:48.630086 791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-0/data/instance:
uuid: "50b46afc138846a184b2b6631eb6d4a1"
format_stamp: "Formatted at 2026-08-10 18:50:48 on dist-test-slave-cbsf"
server_key: "e46424e8a5d6010d53f8c52442a5a92b"
server_key_iv: "df877a16c67adada1b261e26be56bdd8"
server_key_version: "kuduclusterkey@0"
I20260810 18:50:48.630745 791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance:
uuid: "50b46afc138846a184b2b6631eb6d4a1"
format_stamp: "Formatted at 2026-08-10 18:50:48 on dist-test-slave-cbsf"
server_key: "e46424e8a5d6010d53f8c52442a5a92b"
server_key_iv: "df877a16c67adada1b261e26be56bdd8"
server_key_version: "kuduclusterkey@0"
I20260810 18:50:48.636070 791 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260810 18:50:48.661069 810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:48.662830 791 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 18:50:48.663013 791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-0/wal
uuid: "50b46afc138846a184b2b6631eb6d4a1"
format_stamp: "Formatted at 2026-08-10 18:50:48 on dist-test-slave-cbsf"
server_key: "e46424e8a5d6010d53f8c52442a5a92b"
server_key_iv: "df877a16c67adada1b261e26be56bdd8"
server_key_version: "kuduclusterkey@0"
I20260810 18:50:48.663220 791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:50:48.688475 791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:50:48.692298 791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:50:48.692523 791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:50:48.693300 791 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:50:48.694439 791 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:50:48.694541 791 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:48.694636 791 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:50:48.694705 791 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:48.707512 791 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.129:43585
I20260810 18:50:48.708319 936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.129:43585 every 8 connection(s)
I20260810 18:50:48.709309 791 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
I20260810 18:50:48.710908 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 791
I20260810 18:50:48.711048 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance
I20260810 18:50:48.711390 19106 external_mini_cluster.cc:1428] Setting key ce4e0ec28ffc2b2779d2ef0e688f8301
Aug 10 18:50:48 dist-test-slave-cbsf krb5kdc[32616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387848, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.168.130@KRBTEST.COM; defaulting to no policy
I20260810 18:50:48.729070 942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:48.713600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:58567 (local address 127.18.168.190:43801)
0810 18:50:48.715336 (+ 1736us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:48.715342 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:48.715363 (+ 21us) server_negotiation.cc:407] Connection header received
0810 18:50:48.715565 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:48.715569 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:48.715632 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:48.715783 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:50:48.717378 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:48.718282 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:48.720249 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:48.720759 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:48.720887 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:48.724185 (+ 3298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:50:48.724191 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:50:48.724196 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:50:48.724228 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:50:48.726196 (+ 1968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:48.726821 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:48.726824 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:48.726826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:48.726935 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:48.727359 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:48.727363 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:48.727364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:48.727599 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:50:48.727719 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:48.728466 (+ 747us) server_negotiation.cc:299] Negotiation successful
0810 18:50:48.728724 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1647,"thread_start_us":99,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.130@KRBTEST.COM" created.
I20260810 18:50:48.731169 940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:48.712501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43801 (local address 127.18.168.129:58567)
0810 18:50:48.714053 (+ 1552us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:48.714114 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0810 18:50:48.715311 (+ 1197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:50:48.715986 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:50:48.715996 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:50:48.716391 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:50:48.717184 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:48.717209 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:48.718441 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:48.718447 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:48.720082 (+ 1635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:48.720098 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:48.721093 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:48.721100 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:48.721359 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:48.722093 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:50:48.722114 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:50:48.723977 (+ 1863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:50:48.726329 (+ 2352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:48.726334 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:48.726349 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:48.726698 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:48.727048 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:48.727052 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:48.727054 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:48.727236 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:48.727780 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:50:48.727787 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:50:48.728116 (+ 329us) client_negotiation.cc:769] Sending connection context
0810 18:50:48.728334 (+ 218us) client_negotiation.cc:240] Negotiation successful
0810 18:50:48.729937 (+ 1603us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1316,"thread_start_us":184,"threads_started":1}
I20260810 18:50:48.732921 937 heartbeater.cc:347] Connected to a master server at 127.18.168.190:43801
I20260810 18:50:48.733215 937 heartbeater.cc:464] Registering TS with master...
I20260810 18:50:48.733839 937 heartbeater.cc:511] Master 127.18.168.190:43801 requested a full tablet report, sending...
I20260810 18:50:48.735600 698 ts_manager.cc:194] Registered new tserver with Master: 50b46afc138846a184b2b6631eb6d4a1 (127.18.168.129:43585)
I20260810 18:50:48.737193 698 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.129@KRBTEST.COM'} at 127.18.168.129:58567
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab.
Entry for principal kudu/127.18.168.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab.
I20260810 18:50:48.760357 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.168.148:48549/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.130:0
--local_ip_for_outbound_sockets=127.18.168.130
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:43801
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:38753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 18:50:48 dist-test-slave-cbsf krb5kdc[32616](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786387848, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260810 18:50:48.921030 945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:50:48.921387 945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:50:48.921448 945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:50:48.927399 945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:50:48.927472 945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:50:48.927554 945 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.130
I20260810 18:50:48.935274 945 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:38753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.18.168.148:48549/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:43801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.945
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.130
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:50:48.936579 945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:50:48.937955 945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:50:48.950887 952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:50:48.951390 955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:50:48.951463 953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:50:48.952044 945 server_base.cc:1034] running on GCE node
I20260810 18:50:48.952528 945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:50:48.953418 945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:50:48.954632 945 hybrid_clock.cc:648] HybridClock initialized: now 1786387848954602 us; error 42 us; skew 500 ppm
Aug 10 18:50:48 dist-test-slave-cbsf krb5kdc[32616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387848, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:50:48.958652 945 init.cc:377] Logged in from keytab as kudu/127.18.168.130@KRBTEST.COM (short username kudu)
I20260810 18:50:48.959955 945 webserver.cc:460] Webserver started at http://127.18.168.130:43847/ using document root <none> and password file <none>
I20260810 18:50:48.960769 945 fs_manager.cc:359] Metadata directory not provided
I20260810 18:50:48.960906 945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:50:48.961189 945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 18:50:48 dist-test-slave-cbsf krb5kdc[32616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387848, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for HTTP/127.18.168.148@KRBTEST.COM
I20260810 18:50:48.979813 945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-1/data/instance:
uuid: "002006ca84bf445fa0bfc3dae398659a"
format_stamp: "Formatted at 2026-08-10 18:50:48 on dist-test-slave-cbsf"
server_key: "9dc13bc62bb779cec48d08bc33cd6f91"
server_key_iv: "0ece33c9bb6e925a6b647264ce8a487f"
server_key_version: "kuduclusterkey@0"
I20260810 18:50:48.981119 945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance:
uuid: "002006ca84bf445fa0bfc3dae398659a"
format_stamp: "Formatted at 2026-08-10 18:50:48 on dist-test-slave-cbsf"
server_key: "9dc13bc62bb779cec48d08bc33cd6f91"
server_key_iv: "0ece33c9bb6e925a6b647264ce8a487f"
server_key_version: "kuduclusterkey@0"
I20260810 18:50:48.987320 945 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260810 18:50:49.018954 963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:49.020673 945 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260810 18:50:49.020833 945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-1/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-1/wal
uuid: "002006ca84bf445fa0bfc3dae398659a"
format_stamp: "Formatted at 2026-08-10 18:50:48 on dist-test-slave-cbsf"
server_key: "9dc13bc62bb779cec48d08bc33cd6f91"
server_key_iv: "0ece33c9bb6e925a6b647264ce8a487f"
server_key_version: "kuduclusterkey@0"
I20260810 18:50:49.020963 945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:50:49.050534 945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:50:49.054571 945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:50:49.054864 945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:50:49.055683 945 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:50:49.056898 945 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:50:49.057013 945 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:49.057123 945 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:50:49.057214 945 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:49.072456 945 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.130:36225
I20260810 18:50:49.073874 945 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
I20260810 18:50:49.076905 1091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.130:36225 every 8 connection(s)
I20260810 18:50:49.083880 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 945
I20260810 18:50:49.084010 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance
I20260810 18:50:49.084342 19106 external_mini_cluster.cc:1428] Setting key b7eb11ec019d53e4eea7229619e745bb
Aug 10 18:50:49 dist-test-slave-cbsf krb5kdc[32616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387848, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:50:49.095374 942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:49.079303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:45499 (local address 127.18.168.190:43801)
0810 18:50:49.079456 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:49.079545 (+ 89us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:49.080665 (+ 1120us) server_negotiation.cc:407] Connection header received
0810 18:50:49.081987 (+ 1322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:49.081990 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:49.082051 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:49.082152 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:50:49.083642 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:49.084471 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:49.087064 (+ 2593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:49.087407 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:49.087492 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:49.090778 (+ 3286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:50:49.090782 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:50:49.090785 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:50:49.090814 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:50:49.092659 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:49.093396 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:49.093399 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:49.093402 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:49.093514 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:49.093987 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:49.093991 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:49.093992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:49.094220 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:50:49.094323 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:49.095064 (+ 741us) server_negotiation.cc:299] Negotiation successful
0810 18:50:49.095200 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 18:50:49.096477 1094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:49.079851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43801 (local address 127.18.168.130:45499)
0810 18:50:49.080470 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:49.080537 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0810 18:50:49.081715 (+ 1178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:50:49.082362 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:50:49.082371 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:50:49.082772 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:50:49.083451 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:49.083484 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:49.084807 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:49.084811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:49.086917 (+ 2106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:49.086929 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:49.087723 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:49.087727 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:49.087942 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:49.088683 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:50:49.088728 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:50:49.090537 (+ 1809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:50:49.092792 (+ 2255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:49.092801 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:49.092822 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:49.093248 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:49.093631 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:49.093638 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:49.093642 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:49.093855 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:49.094348 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:50:49.094357 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:50:49.094733 (+ 376us) client_negotiation.cc:769] Sending connection context
0810 18:50:49.094970 (+ 237us) client_negotiation.cc:240] Negotiation successful
0810 18:50:49.095362 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":174,"threads_started":1}
I20260810 18:50:49.097940 1092 heartbeater.cc:347] Connected to a master server at 127.18.168.190:43801
I20260810 18:50:49.098201 1092 heartbeater.cc:464] Registering TS with master...
I20260810 18:50:49.098773 1092 heartbeater.cc:511] Master 127.18.168.190:43801 requested a full tablet report, sending...
I20260810 18:50:49.100026 698 ts_manager.cc:194] Registered new tserver with Master: 002006ca84bf445fa0bfc3dae398659a (127.18.168.130:36225)
I20260810 18:50:49.100823 698 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.130@KRBTEST.COM'} at 127.18.168.130:45499
WARNING: no policy specified for kudu/127.18.168.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab.
Entry for principal kudu/127.18.168.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab.
I20260810 18:50:49.136284 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.168.148:48549/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.131:0
--local_ip_for_outbound_sockets=127.18.168.131
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:43801
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:38753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:50:49.305891 1097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:50:49.306299 1097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:50:49.306401 1097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:50:49.312479 1097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:50:49.312644 1097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:50:49.312801 1097 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.131
I20260810 18:50:49.321116 1097 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:38753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.18.168.148:48549/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:43801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.131
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:50:49.322728 1097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:50:49.324122 1097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
Aug 10 18:50:49 dist-test-slave-cbsf krb5kdc[32616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387848, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for HTTP/127.18.168.148@KRBTEST.COM
W20260810 18:50:49.348982 1105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:50:49.348999 1108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:50:49.350256 1097 server_base.cc:1034] running on GCE node
W20260810 18:50:49.350472 1104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:50:49.351019 1097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:50:49.351871 1097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:50:49.359287 1097 hybrid_clock.cc:648] HybridClock initialized: now 1786387849359198 us; error 100 us; skew 500 ppm
Aug 10 18:50:49 dist-test-slave-cbsf krb5kdc[32616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387849, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:50:49.362846 1097 init.cc:377] Logged in from keytab as kudu/127.18.168.131@KRBTEST.COM (short username kudu)
I20260810 18:50:49.363927 1097 webserver.cc:460] Webserver started at http://127.18.168.131:41575/ using document root <none> and password file <none>
I20260810 18:50:49.364583 1097 fs_manager.cc:359] Metadata directory not provided
I20260810 18:50:49.364665 1097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:50:49.364915 1097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 18:50:49 dist-test-slave-cbsf krb5kdc[32616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387849, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for HTTP/127.18.168.148@KRBTEST.COM
I20260810 18:50:49.385632 1097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-2/data/instance:
uuid: "115c47de96be486f9f6e0554b6724f15"
format_stamp: "Formatted at 2026-08-10 18:50:49 on dist-test-slave-cbsf"
server_key: "1955e48402cd2db4f08e3cc15a37f7ec"
server_key_iv: "1c51b0aca4926d40a679648347607368"
server_key_version: "kuduclusterkey@0"
I20260810 18:50:49.386421 1097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance:
uuid: "115c47de96be486f9f6e0554b6724f15"
format_stamp: "Formatted at 2026-08-10 18:50:49 on dist-test-slave-cbsf"
server_key: "1955e48402cd2db4f08e3cc15a37f7ec"
server_key_iv: "1c51b0aca4926d40a679648347607368"
server_key_version: "kuduclusterkey@0"
I20260810 18:50:49.392042 1097 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 18:50:49.424309 1115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:49.426082 1097 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.004s sys 0.000s
I20260810 18:50:49.426237 1097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-2/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-2/wal
uuid: "115c47de96be486f9f6e0554b6724f15"
format_stamp: "Formatted at 2026-08-10 18:50:49 on dist-test-slave-cbsf"
server_key: "1955e48402cd2db4f08e3cc15a37f7ec"
server_key_iv: "1c51b0aca4926d40a679648347607368"
server_key_version: "kuduclusterkey@0"
I20260810 18:50:49.426380 1097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:50:49.468313 1097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:50:49.472193 1097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:50:49.472432 1097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:50:49.473184 1097 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:50:49.474696 1097 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:50:49.474798 1097 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:49.474895 1097 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:50:49.474964 1097 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:49.489585 1097 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.131:44051
I20260810 18:50:49.491020 1097 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
I20260810 18:50:49.494678 1243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.131:44051 every 8 connection(s)
I20260810 18:50:49.500535 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 1097
I20260810 18:50:49.500664 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance
I20260810 18:50:49.501016 19106 external_mini_cluster.cc:1428] Setting key 337fceae28e7079edaa416eb701dddc6
Aug 10 18:50:49 dist-test-slave-cbsf krb5kdc[32616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387849, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:50:49.520509 942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:49.496883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:50395 (local address 127.18.168.190:43801)
0810 18:50:49.497013 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:49.497018 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:49.505235 (+ 8217us) server_negotiation.cc:407] Connection header received
0810 18:50:49.506482 (+ 1247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:49.506485 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:49.506537 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:49.506586 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:50:49.508262 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:49.509082 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:49.510743 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:49.511097 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:49.511208 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:49.515897 (+ 4689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:50:49.515901 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:50:49.515904 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:50:49.515930 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:50:49.518208 (+ 2278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:49.518807 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:49.518811 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:49.518812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:49.518895 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:49.519306 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:49.519310 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:49.519311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:49.519517 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:50:49.519585 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:49.520242 (+ 657us) server_negotiation.cc:299] Negotiation successful
0810 18:50:49.520356 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 18:50:49.522632 1246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:49.497481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43801 (local address 127.18.168.131:50395)
0810 18:50:49.505072 (+ 7591us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:49.505138 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0810 18:50:49.506228 (+ 1090us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:50:49.507026 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:50:49.507036 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:50:49.507410 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:50:49.508105 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:49.508124 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:49.509237 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:49.509240 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:49.510606 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:49.510617 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:49.511451 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:49.511454 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:49.511652 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:49.512337 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:50:49.512358 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:50:49.515714 (+ 3356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:50:49.518348 (+ 2634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:49.518353 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:49.518371 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:49.518694 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:49.519015 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:49.519018 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:49.519020 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:49.519188 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:49.519664 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:50:49.519670 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:50:49.519976 (+ 306us) client_negotiation.cc:769] Sending connection context
0810 18:50:49.521212 (+ 1236us) client_negotiation.cc:240] Negotiation successful
0810 18:50:49.521556 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7324,"thread_start_us":143,"threads_started":1}
I20260810 18:50:49.524123 1244 heartbeater.cc:347] Connected to a master server at 127.18.168.190:43801
I20260810 18:50:49.524367 1244 heartbeater.cc:464] Registering TS with master...
I20260810 18:50:49.524937 1244 heartbeater.cc:511] Master 127.18.168.190:43801 requested a full tablet report, sending...
I20260810 18:50:49.527560 698 ts_manager.cc:194] Registered new tserver with Master: 115c47de96be486f9f6e0554b6724f15 (127.18.168.131:44051)
I20260810 18:50:49.528280 698 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.131@KRBTEST.COM'} at 127.18.168.131:50395
I20260810 18:50:49.539211 19106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:50:49.540072 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 791
I20260810 18:50:49.567766 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 945
I20260810 18:50:49.586659 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 1097
I20260810 18:50:49.595932 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 661
I20260810 18:50:49.609117 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:43801
--webserver_interface=127.18.168.190
--webserver_port=38075
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:38753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:43801
--ranger_config_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.168.148:48549/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.168.148:48549/kms with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:50:49.721632 1248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:50:49.721895 1248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:50:49.721935 1248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:50:49.725950 1248 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:50:49.726004 1248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:50:49.726023 1248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:50:49.726039 1248 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:50:49.726089 1248 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:50:49.731453 1248 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:38753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.168.148:48549/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:43801
--ranger_config_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:43801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=38075
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:50:49.732481 1248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:50:49.733510 1248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:50:49.742069 1254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:50:49.742092 1253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:50:49.742391 1248 server_base.cc:1034] running on GCE node
W20260810 18:50:49.742095 1256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:50:49.742990 1248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:50:49.744103 1248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:50:49.745267 1248 hybrid_clock.cc:648] HybridClock initialized: now 1786387849745265 us; error 38 us; skew 500 ppm
Aug 10 18:50:49 dist-test-slave-cbsf krb5kdc[32616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387849, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:50:49.747938 1248 init.cc:377] Logged in from keytab as kudu/127.18.168.190@KRBTEST.COM (short username kudu)
I20260810 18:50:49.748872 1248 webserver.cc:460] Webserver started at http://127.18.168.190:38075/ using document root <none> and password file <none>
I20260810 18:50:49.749392 1248 fs_manager.cc:359] Metadata directory not provided
I20260810 18:50:49.749483 1248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 18:50:49 dist-test-slave-cbsf krb5kdc[32616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387849, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for HTTP/127.18.168.148@KRBTEST.COM
I20260810 18:50:49.826683 1248 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 18:50:49.829507 1265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:49.831090 1248 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:50:49.831233 1248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "db5a963396c94f9ca5b846feadc654f7"
format_stamp: "Formatted at 2026-08-10 18:50:47 on dist-test-slave-cbsf"
server_key: "96281fc55bea126fbcd2e3a59ae4082a"
server_key_iv: "128e051848ee13d0aff34a455e64e65c"
server_key_version: "kuduclusterkey@0"
I20260810 18:50:49.831831 1248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:50:49.852723 1248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:50:49.855180 1248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:50:49.855335 1248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:50:49.862792 1248 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:43801
I20260810 18:50:49.862798 1325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:43801 every 8 connection(s)
I20260810 18:50:49.863871 1248 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:50:49.867504 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 1248
I20260810 18:50:49.867966 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.168.148:48549/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.129:43585
--local_ip_for_outbound_sockets=127.18.168.129
--tserver_master_addrs=127.18.168.190:43801
--webserver_port=45999
--webserver_interface=127.18.168.129
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:38753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.168.148:48549/kms with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:50:49.868505 1326 sys_catalog.cc:263] Verifying existing consensus state
I20260810 18:50:49.870175 1326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7: Bootstrap starting.
I20260810 18:50:49.886582 1326 log.cc:824] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7: Log is configured to *not* fsync() on all Append() calls
I20260810 18:50:49.893198 1326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:50:49.893584 1326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7: Bootstrap complete.
I20260810 18:50:49.897718 1326 raft_consensus.cc:348] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5a963396c94f9ca5b846feadc654f7" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43801 } }
I20260810 18:50:49.898116 1326 raft_consensus.cc:729] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: db5a963396c94f9ca5b846feadc654f7, State: Initialized, Role: FOLLOWER
I20260810 18:50:49.898648 1326 consensus_queue.cc:260] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5a963396c94f9ca5b846feadc654f7" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43801 } }
I20260810 18:50:49.898789 1326 raft_consensus.cc:388] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:50:49.898872 1326 raft_consensus.cc:482] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:50:49.898985 1326 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7 [term 1 FOLLOWER]: Advancing to term 2
I20260810 18:50:49.900797 1326 raft_consensus.cc:504] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5a963396c94f9ca5b846feadc654f7" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43801 } }
I20260810 18:50:49.901182 1326 leader_election.cc:302] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db5a963396c94f9ca5b846feadc654f7; no voters:
I20260810 18:50:49.901504 1326 leader_election.cc:288] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 18:50:49.901678 1330 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 18:50:49.901939 1330 raft_consensus.cc:686] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7 [term 2 LEADER]: Becoming Leader. State: Replica: db5a963396c94f9ca5b846feadc654f7, State: Running, Role: LEADER
I20260810 18:50:49.902244 1330 consensus_queue.cc:237] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5a963396c94f9ca5b846feadc654f7" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43801 } }
I20260810 18:50:49.902755 1326 sys_catalog.cc:564] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:50:49.903759 1332 sys_catalog.cc:455] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db5a963396c94f9ca5b846feadc654f7. Latest consensus state: current_term: 2 leader_uuid: "db5a963396c94f9ca5b846feadc654f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5a963396c94f9ca5b846feadc654f7" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43801 } } }
I20260810 18:50:49.903810 1331 sys_catalog.cc:455] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "db5a963396c94f9ca5b846feadc654f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5a963396c94f9ca5b846feadc654f7" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43801 } } }
I20260810 18:50:49.903888 1332 sys_catalog.cc:458] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7 [sys.catalog]: This master's current role is: LEADER
I20260810 18:50:49.903915 1331 sys_catalog.cc:458] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7 [sys.catalog]: This master's current role is: LEADER
I20260810 18:50:49.904417 1340 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:50:49.907733 1340 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:50:49.908215 1326 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 18:50:49.908372 1340 catalog_manager.cc:1259] Loaded cluster ID: f0f42adc87fe4d91b3f7a4dbae6fefe0
I20260810 18:50:49.908435 1340 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:50:49.913164 1340 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:50:49.914073 1340 catalog_manager.cc:5879] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7: Loaded TSK: 0
I20260810 18:50:49.915560 1340 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 18:50:50.008322 1328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:50:50.008580 1328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:50:50.008656 1328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:50:50.013881 1328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:50:50.013963 1328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:50:50.014046 1328 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.129
I20260810 18:50:50.020462 1328 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:38753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.168.148:48549/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.129:43585
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.168.129
--webserver_port=45999
--tserver_master_addrs=127.18.168.190:43801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.129
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:50:50.021621 1328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:50:50.022625 1328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:50:50.032846 1369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:50:50.033704 1328 server_base.cc:1034] running on GCE node
W20260810 18:50:50.033871 1370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:50:50.033705 1373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:50:50.034363 1328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:50:50.035161 1328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:50:50.036371 1328 hybrid_clock.cc:648] HybridClock initialized: now 1786387850036328 us; error 45 us; skew 500 ppm
Aug 10 18:50:50 dist-test-slave-cbsf krb5kdc[32616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387850, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:50:50.043254 1328 init.cc:377] Logged in from keytab as kudu/127.18.168.129@KRBTEST.COM (short username kudu)
I20260810 18:50:50.044343 1328 webserver.cc:460] Webserver started at http://127.18.168.129:45999/ using document root <none> and password file <none>
I20260810 18:50:50.044994 1328 fs_manager.cc:359] Metadata directory not provided
I20260810 18:50:50.045089 1328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 18:50:50 dist-test-slave-cbsf krb5kdc[32616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387850, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for HTTP/127.18.168.148@KRBTEST.COM
I20260810 18:50:50.073385 1328 fs_manager.cc:709] Time spent opening directory manager: real 0.010s user 0.003s sys 0.000s
I20260810 18:50:50.076845 1382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:50.078527 1328 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 18:50:50.078648 1328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-0/wal
uuid: "50b46afc138846a184b2b6631eb6d4a1"
format_stamp: "Formatted at 2026-08-10 18:50:48 on dist-test-slave-cbsf"
server_key: "e46424e8a5d6010d53f8c52442a5a92b"
server_key_iv: "df877a16c67adada1b261e26be56bdd8"
server_key_version: "kuduclusterkey@0"
I20260810 18:50:50.079331 1328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:50:50.109017 1328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:50:50.112741 1328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:50:50.112991 1328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:50:50.113725 1328 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:50:50.114835 1328 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:50:50.114944 1328 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:50.115016 1328 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:50:50.115048 1328 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:50.127722 1328 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.129:43585
I20260810 18:50:50.130119 1512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.129:43585 every 8 connection(s)
I20260810 18:50:50.131220 1328 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
I20260810 18:50:50.144238 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 1328
I20260810 18:50:50.144833 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.168.148:48549/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.130:36225
--local_ip_for_outbound_sockets=127.18.168.130
--tserver_master_addrs=127.18.168.190:43801
--webserver_port=43847
--webserver_interface=127.18.168.130
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:38753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.168.148:48549/kms with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 18:50:50 dist-test-slave-cbsf krb5kdc[32616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387850, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:50:50.153831 1516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:50.133901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43801 (local address 127.18.168.129:46795)
0810 18:50:50.135479 (+ 1578us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:50.135537 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 18:50:50.136625 (+ 1088us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:50:50.138176 (+ 1551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:50:50.138186 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:50:50.138566 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:50:50.139454 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:50.139476 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:50.141469 (+ 1993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:50.141473 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:50.142868 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:50.142881 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:50.144210 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:50.144215 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:50.144407 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:50.145143 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:50:50.145184 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:50:50.147000 (+ 1816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:50:50.149725 (+ 2725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:50.149733 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:50.149747 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:50.150066 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:50.150492 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:50.150497 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:50.150500 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:50.150716 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:50.151673 (+ 957us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:50:50.151680 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:50:50.152009 (+ 329us) client_negotiation.cc:769] Sending connection context
0810 18:50:50.152236 (+ 227us) client_negotiation.cc:240] Negotiation successful
0810 18:50:50.152612 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1341,"thread_start_us":199,"threads_started":1}
I20260810 18:50:50.155866 1515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:50.133822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:46795 (local address 127.18.168.190:43801)
0810 18:50:50.135156 (+ 1334us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:50.135182 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:50.135690 (+ 508us) server_negotiation.cc:407] Connection header received
0810 18:50:50.137210 (+ 1520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:50.137226 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:50.137682 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:50.138062 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:50:50.139668 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:50.141291 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:50.143045 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:50.143783 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:50.143979 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:50.147238 (+ 3259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:50:50.147247 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:50:50.147263 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:50:50.147296 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:50:50.149602 (+ 2306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:50.150204 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:50.150213 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:50.150221 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:50.150371 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:50.150885 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:50.150908 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:50.150911 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:50.151445 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:50:50.151609 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:50.154482 (+ 2873us) server_negotiation.cc:299] Negotiation successful
0810 18:50:50.154793 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1143,"thread_start_us":160,"threads_started":1}
I20260810 18:50:50.159144 1513 heartbeater.cc:347] Connected to a master server at 127.18.168.190:43801
I20260810 18:50:50.159412 1513 heartbeater.cc:464] Registering TS with master...
I20260810 18:50:50.160075 1513 heartbeater.cc:511] Master 127.18.168.190:43801 requested a full tablet report, sending...
W20260810 18:50:50.162176 1513 heartbeater.cc:644] Failed to heartbeat to 127.18.168.190:43801 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 18:50:50.162302 1513 heartbeater.cc:464] Registering TS with master...
I20260810 18:50:50.162465 1513 heartbeater.cc:511] Master 127.18.168.190:43801 requested a full tablet report, sending...
I20260810 18:50:50.163432 1513 heartbeater.cc:464] Registering TS with master...
I20260810 18:50:50.163617 1513 heartbeater.cc:511] Master 127.18.168.190:43801 requested a full tablet report, sending...
W20260810 18:50:50.164379 1513 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 18:50:50.301615 1518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:50:50.301906 1518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:50:50.301992 1518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:50:50.306303 1518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:50:50.306389 1518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:50:50.306483 1518 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.130
I20260810 18:50:50.311662 1518 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:38753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.168.148:48549/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.130:36225
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.168.130
--webserver_port=43847
--tserver_master_addrs=127.18.168.190:43801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.130
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:50:50.312662 1518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:50:50.313786 1518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:50:50.323980 1523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:50:50.324081 1526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:50:50.324290 1524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:50:50.324770 1518 server_base.cc:1034] running on GCE node
I20260810 18:50:50.325168 1518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:50:50.325771 1518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:50:50.326956 1518 hybrid_clock.cc:648] HybridClock initialized: now 1786387850326930 us; error 44 us; skew 500 ppm
Aug 10 18:50:50 dist-test-slave-cbsf krb5kdc[32616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387850, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:50:50.330327 1518 init.cc:377] Logged in from keytab as kudu/127.18.168.130@KRBTEST.COM (short username kudu)
I20260810 18:50:50.331555 1518 webserver.cc:460] Webserver started at http://127.18.168.130:43847/ using document root <none> and password file <none>
I20260810 18:50:50.332298 1518 fs_manager.cc:359] Metadata directory not provided
I20260810 18:50:50.332419 1518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 18:50:50 dist-test-slave-cbsf krb5kdc[32616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387850, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for HTTP/127.18.168.148@KRBTEST.COM
I20260810 18:50:50.352751 1518 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:50:50.357004 1535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:50.358388 1518 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260810 18:50:50.358560 1518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-1/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-1/wal
uuid: "002006ca84bf445fa0bfc3dae398659a"
format_stamp: "Formatted at 2026-08-10 18:50:48 on dist-test-slave-cbsf"
server_key: "9dc13bc62bb779cec48d08bc33cd6f91"
server_key_iv: "0ece33c9bb6e925a6b647264ce8a487f"
server_key_version: "kuduclusterkey@0"
I20260810 18:50:50.359105 1518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:50:50.384370 1518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:50:50.388317 1518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:50:50.388538 1518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:50:50.389302 1518 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:50:50.390439 1518 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:50:50.390504 1518 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:50.390589 1518 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:50:50.390623 1518 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:50.401564 1518 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.130:36225
I20260810 18:50:50.401620 1668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.130:36225 every 8 connection(s)
I20260810 18:50:50.403062 1518 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
I20260810 18:50:50.404743 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 1518
I20260810 18:50:50.405260 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.168.148:48549/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.131:44051
--local_ip_for_outbound_sockets=127.18.168.131
--tserver_master_addrs=127.18.168.190:43801
--webserver_port=41575
--webserver_interface=127.18.168.131
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:38753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.168.148:48549/kms with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 18:50:50 dist-test-slave-cbsf krb5kdc[32616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387850, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:50:50.420840 1515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:50.405358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:47223 (local address 127.18.168.190:43801)
0810 18:50:50.407443 (+ 2085us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:50.407448 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:50.407466 (+ 18us) server_negotiation.cc:407] Connection header received
0810 18:50:50.407778 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:50.407782 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:50.407833 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:50.407923 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:50:50.409500 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:50.410328 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:50.412050 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:50.412415 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:50.412508 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:50.415817 (+ 3309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:50:50.415822 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:50:50.415825 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:50:50.415855 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:50:50.418081 (+ 2226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:50.418768 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:50.418772 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:50.418774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:50.418878 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:50.419361 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:50.419365 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:50.419367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:50.419601 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:50:50.419708 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:50.420474 (+ 766us) server_negotiation.cc:299] Negotiation successful
0810 18:50:50.420616 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1988}
I20260810 18:50:50.422515 1671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:50.405800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43801 (local address 127.18.168.130:47223)
0810 18:50:50.406380 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:50.406440 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 18:50:50.407535 (+ 1095us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:50:50.408152 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:50:50.408162 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:50:50.408552 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:50:50.409297 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:50.409319 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:50.410537 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:50.410540 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:50.411878 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:50.411891 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:50.412750 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:50.412756 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:50.412985 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:50.413724 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:50:50.413747 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:50:50.415603 (+ 1856us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:50:50.418236 (+ 2633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:50.418243 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:50.418259 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:50.418628 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:50.419033 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:50.419036 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:50.419039 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:50.419221 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:50.419802 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:50:50.419808 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:50:50.420159 (+ 351us) client_negotiation.cc:769] Sending connection context
0810 18:50:50.421094 (+ 935us) client_negotiation.cc:240] Negotiation successful
0810 18:50:50.421478 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":156,"threads_started":1}
I20260810 18:50:50.424053 1669 heartbeater.cc:347] Connected to a master server at 127.18.168.190:43801
I20260810 18:50:50.424307 1669 heartbeater.cc:464] Registering TS with master...
I20260810 18:50:50.424996 1669 heartbeater.cc:511] Master 127.18.168.190:43801 requested a full tablet report, sending...
W20260810 18:50:50.426803 1669 heartbeater.cc:644] Failed to heartbeat to 127.18.168.190:43801 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 18:50:50.426952 1669 heartbeater.cc:464] Registering TS with master...
I20260810 18:50:50.427141 1669 heartbeater.cc:511] Master 127.18.168.190:43801 requested a full tablet report, sending...
I20260810 18:50:50.428033 1669 heartbeater.cc:464] Registering TS with master...
I20260810 18:50:50.428221 1669 heartbeater.cc:511] Master 127.18.168.190:43801 requested a full tablet report, sending...
W20260810 18:50:50.429059 1669 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 18:50:50.546996 1673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:50:50.547313 1673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:50:50.547391 1673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:50:50.551609 1673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:50:50.551687 1673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:50:50.551770 1673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.131
I20260810 18:50:50.558439 1673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:38753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.168.148:48549/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.131:44051
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.168.131
--webserver_port=41575
--tserver_master_addrs=127.18.168.190:43801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.131
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:50:50.559767 1673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:50:50.560963 1673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:50:50.571574 1683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:50:50.571574 1680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:50:50.571658 1681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:50:50.572062 1673 server_base.cc:1034] running on GCE node
I20260810 18:50:50.572499 1673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:50:50.573215 1673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:50:50.574607 1673 hybrid_clock.cc:648] HybridClock initialized: now 1786387850574583 us; error 36 us; skew 500 ppm
Aug 10 18:50:50 dist-test-slave-cbsf krb5kdc[32616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387850, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:50:50.577912 1673 init.cc:377] Logged in from keytab as kudu/127.18.168.131@KRBTEST.COM (short username kudu)
I20260810 18:50:50.579013 1673 webserver.cc:460] Webserver started at http://127.18.168.131:41575/ using document root <none> and password file <none>
I20260810 18:50:50.579748 1673 fs_manager.cc:359] Metadata directory not provided
I20260810 18:50:50.579849 1673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 18:50:50 dist-test-slave-cbsf krb5kdc[32616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387850, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for HTTP/127.18.168.148@KRBTEST.COM
I20260810 18:50:50.609647 1673 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.003s sys 0.000s
I20260810 18:50:50.612315 1691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:50.614046 1673 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:50:50.614190 1673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-2/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-2/wal
uuid: "115c47de96be486f9f6e0554b6724f15"
format_stamp: "Formatted at 2026-08-10 18:50:49 on dist-test-slave-cbsf"
server_key: "1955e48402cd2db4f08e3cc15a37f7ec"
server_key_iv: "1c51b0aca4926d40a679648347607368"
server_key_version: "kuduclusterkey@0"
I20260810 18:50:50.614930 1673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:50:50.636654 1673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:50:50.640498 1673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:50:50.640731 1673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:50:50.641506 1673 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:50:50.642640 1673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:50:50.642719 1673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:50.642798 1673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:50:50.642838 1673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:50.655315 1673 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.131:44051
I20260810 18:50:50.655385 1823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.131:44051 every 8 connection(s)
I20260810 18:50:50.656662 1673 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
I20260810 18:50:50.657466 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 1673
I20260810 18:50:50.671068 1515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:50.660357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59954 (local address 127.18.168.190:43801)
0810 18:50:50.660538 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:50.660545 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:50.661401 (+ 856us) server_negotiation.cc:407] Connection header received
0810 18:50:50.661673 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:50.661677 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:50.661734 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:50.661799 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:50:50.663155 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:50.663966 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:50.665253 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:50.665480 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:50.665585 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:50.667508 (+ 1923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:50:50.667513 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:50:50.667515 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:50:50.667541 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:50:50.669371 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:50.669823 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:50.669828 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:50.669830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:50.669914 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:50.670226 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:50.670230 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:50.670232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:50.670403 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:50:50.670608 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:50.670760 (+ 152us) server_negotiation.cc:299] Negotiation successful
0810 18:50:50.670884 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
Aug 10 18:50:50 dist-test-slave-cbsf krb5kdc[32616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387850, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:50:50.680860 1828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:50.663399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:56305 (local address 127.18.168.190:43801)
0810 18:50:50.664215 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:50.664219 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:50.665848 (+ 1629us) server_negotiation.cc:407] Connection header received
0810 18:50:50.667130 (+ 1282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:50.667133 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:50.667183 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:50.667268 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:50:50.669212 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:50.670001 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:50.673587 (+ 3586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:50.673957 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:50.674099 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:50.676392 (+ 2293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:50:50.676397 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:50:50.676400 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:50:50.676440 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:50:50.678295 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:50.679008 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:50.679012 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:50.679014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:50.679115 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:50.679583 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:50.679588 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:50.679590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:50.679823 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:50:50.679942 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:50.680482 (+ 540us) server_negotiation.cc:299] Negotiation successful
0810 18:50:50.680618 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":744,"thread_start_us":110,"threads_started":1}
I20260810 18:50:50.681679 1827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:50.662360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43801 (local address 127.18.168.131:56305)
0810 18:50:50.665195 (+ 2835us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:50.665766 (+ 571us) client_negotiation.cc:174] Beginning negotiation
0810 18:50:50.666873 (+ 1107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:50:50.667814 (+ 941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:50:50.667827 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:50:50.668243 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:50:50.669044 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:50.669066 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:50.670176 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:50.670180 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:50.673440 (+ 3260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:50.673452 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:50.674127 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:50.674130 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:50.674272 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:50.674801 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:50:50.674817 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:50:50.676187 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:50:50.678452 (+ 2265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:50.678458 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:50.678474 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:50.678847 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:50.679250 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:50.679254 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:50.679256 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:50.679445 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:50.679960 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:50:50.679965 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:50:50.680202 (+ 237us) client_negotiation.cc:769] Sending connection context
0810 18:50:50.680354 (+ 152us) client_negotiation.cc:240] Negotiation successful
0810 18:50:50.680588 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2609,"thread_start_us":128,"threads_started":1}
I20260810 18:50:50.683079 1824 heartbeater.cc:347] Connected to a master server at 127.18.168.190:43801
I20260810 18:50:50.683333 1824 heartbeater.cc:464] Registering TS with master...
I20260810 18:50:50.683923 1824 heartbeater.cc:511] Master 127.18.168.190:43801 requested a full tablet report, sending...
W20260810 18:50:50.689548 1824 heartbeater.cc:644] Failed to heartbeat to 127.18.168.190:43801 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 18:50:50.689694 1824 heartbeater.cc:464] Registering TS with master...
I20260810 18:50:50.689877 1824 heartbeater.cc:511] Master 127.18.168.190:43801 requested a full tablet report, sending...
I20260810 18:50:50.690843 1824 heartbeater.cc:464] Registering TS with master...
I20260810 18:50:50.691021 1824 heartbeater.cc:511] Master 127.18.168.190:43801 requested a full tablet report, sending...
W20260810 18:50:50.691943 1824 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 10 18:50:51 dist-test-slave-cbsf krb5kdc[32616](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786387851, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:50:51.164605 1513 heartbeater.cc:464] Registering TS with master...
I20260810 18:50:51.164930 1513 heartbeater.cc:511] Master 127.18.168.190:43801 requested a full tablet report, sending...
I20260810 18:50:51.166206 1285 ts_manager.cc:194] Registered new tserver with Master: 50b46afc138846a184b2b6631eb6d4a1 (127.18.168.129:43585)
I20260810 18:50:51.167488 1285 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.129@KRBTEST.COM'} at 127.18.168.129:46795
I20260810 18:50:51.429267 1669 heartbeater.cc:464] Registering TS with master...
I20260810 18:50:51.429600 1669 heartbeater.cc:511] Master 127.18.168.190:43801 requested a full tablet report, sending...
I20260810 18:50:51.430344 1285 ts_manager.cc:194] Registered new tserver with Master: 002006ca84bf445fa0bfc3dae398659a (127.18.168.130:36225)
I20260810 18:50:51.431129 1285 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.130@KRBTEST.COM'} at 127.18.168.130:47223
Aug 10 18:50:51 dist-test-slave-cbsf krb5kdc[32616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387851, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for HTTP/127.18.168.148@KRBTEST.COM
Aug 10 18:50:51 dist-test-slave-cbsf krb5kdc[32616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387851, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for HTTP/127.18.168.148@KRBTEST.COM
I20260810 18:50:51.692241 1824 heartbeater.cc:464] Registering TS with master...
I20260810 18:50:51.692536 1824 heartbeater.cc:511] Master 127.18.168.190:43801 requested a full tablet report, sending...
I20260810 18:50:51.693464 1285 ts_manager.cc:194] Registered new tserver with Master: 115c47de96be486f9f6e0554b6724f15 (127.18.168.131:44051)
I20260810 18:50:51.694257 1285 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.131@KRBTEST.COM'} at 127.18.168.131:56305
I20260810 18:50:51.694626 19106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:50:51.710417 1850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:51.700301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59970 (local address 127.18.168.190:43801)
0810 18:50:51.700607 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:51.700612 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:51.700680 (+ 68us) server_negotiation.cc:407] Connection header received
0810 18:50:51.700837 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:51.700840 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:51.700892 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:51.700985 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:50:51.701805 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:51.702556 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:51.703830 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:51.704352 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:51.704506 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:51.706265 (+ 1759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:50:51.706269 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:50:51.706272 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:50:51.706300 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:50:51.708128 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:51.708585 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:51.708589 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:51.708590 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:51.708740 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:51.709042 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:51.709046 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:51.709048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:51.709271 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:50:51.709415 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:51.710138 (+ 723us) server_negotiation.cc:299] Negotiation successful
0810 18:50:51.710258 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":122,"threads_started":1}
I20260810 18:50:51.720451 1285 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59970:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:50:51.724455 1285 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:50:51.754267 1859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:51.742284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:36225 (local address 127.0.0.1:51410)
0810 18:50:51.747494 (+ 5210us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:51.747546 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 18:50:51.747682 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:50:51.748655 (+ 973us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:50:51.748662 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:50:51.748686 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:50:51.748866 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:51.748874 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:51.750943 (+ 2069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:51.750946 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:51.752339 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:51.752368 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:51.753711 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:51.753714 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:51.753827 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:51.753864 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 18:50:51.753983 (+ 119us) client_negotiation.cc:240] Negotiation successful
0810 18:50:51.754104 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5036,"thread_start_us":130,"threads_started":1}
I20260810 18:50:51.758908 1858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:51.743205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51410 (local address 127.18.168.130:36225)
0810 18:50:51.747961 (+ 4756us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:51.747968 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:51.748012 (+ 44us) server_negotiation.cc:407] Connection header received
0810 18:50:51.748093 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:51.748099 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:51.748320 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:51.748465 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:50:51.749015 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:51.750816 (+ 1801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:51.752497 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:51.753390 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:51.753512 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:51.753646 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:51.758659 (+ 5013us) server_negotiation.cc:299] Negotiation successful
0810 18:50:51.758767 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4642,"thread_start_us":91,"threads_started":1}
I20260810 18:50:51.762581 1594 tablet_service.cc:1467] Processing CreateTablet for tablet b9036e4ad8604efab1bcaf16e91d3a05 (DEFAULT_TABLE table=test-table [id=545e80d005e349c28170569925e976d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:50:51.764001 1594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9036e4ad8604efab1bcaf16e91d3a05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:50:51.769307 1857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:51.743626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.131:44051 (local address 127.0.0.1:50256)
0810 18:50:51.748864 (+ 5238us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:51.748879 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:50:51.748980 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:50:51.749683 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:50:51.749685 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:50:51.749722 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:50:51.749814 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:51.749824 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:51.766477 (+ 16653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:51.766482 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:51.767553 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:51.767566 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:51.768853 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:51.768857 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:51.768962 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:51.768995 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 18:50:51.769057 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 18:50:51.769161 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5161,"thread_start_us":90,"threads_started":1}
I20260810 18:50:51.770701 1860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:51.743782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50256 (local address 127.18.168.131:44051)
0810 18:50:51.747245 (+ 3463us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:51.747252 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:51.749083 (+ 1831us) server_negotiation.cc:407] Connection header received
0810 18:50:51.749179 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:51.749184 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:51.749360 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:51.749497 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:50:51.749950 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:51.766353 (+ 16403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:51.767724 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:51.768672 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:51.770252 (+ 1580us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:51.770403 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:51.770461 (+ 58us) server_negotiation.cc:299] Negotiation successful
0810 18:50:51.770559 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3359,"thread_start_us":87,"threads_started":1}
I20260810 18:50:51.773818 1750 tablet_service.cc:1467] Processing CreateTablet for tablet b9036e4ad8604efab1bcaf16e91d3a05 (DEFAULT_TABLE table=test-table [id=545e80d005e349c28170569925e976d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:50:51.775035 1750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9036e4ad8604efab1bcaf16e91d3a05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:50:51.779381 1859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:51.745111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:43585 (local address 127.0.0.1:51022)
0810 18:50:51.754896 (+ 9785us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:51.754909 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:50:51.755064 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:50:51.755752 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:50:51.755755 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:50:51.755772 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:50:51.755841 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:51.755847 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:51.757347 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:51.757350 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:51.758489 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:51.758500 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:51.778878 (+ 20378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:51.778882 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:51.779004 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:51.779029 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 18:50:51.779108 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 18:50:51.779220 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9723,"thread_start_us":93,"threads_started":1}
I20260810 18:50:51.779888 1862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:51.745266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51022 (local address 127.18.168.129:43585)
0810 18:50:51.746968 (+ 1702us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:51.746975 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:51.754992 (+ 8017us) server_negotiation.cc:407] Connection header received
0810 18:50:51.755217 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:51.755241 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:51.755436 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:51.755571 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:50:51.755992 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:51.757194 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:51.776180 (+ 18986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:51.778504 (+ 2324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:51.778617 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:51.778765 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:51.779650 (+ 885us) server_negotiation.cc:299] Negotiation successful
0810 18:50:51.779755 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1573,"thread_start_us":94,"threads_started":1}
I20260810 18:50:51.785930 1437 tablet_service.cc:1467] Processing CreateTablet for tablet b9036e4ad8604efab1bcaf16e91d3a05 (DEFAULT_TABLE table=test-table [id=545e80d005e349c28170569925e976d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:50:51.787051 1437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9036e4ad8604efab1bcaf16e91d3a05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:50:51.798641 1867 tablet_bootstrap.cc:492] T b9036e4ad8604efab1bcaf16e91d3a05 P 50b46afc138846a184b2b6631eb6d4a1: Bootstrap starting.
I20260810 18:50:51.801338 1867 tablet_bootstrap.cc:654] T b9036e4ad8604efab1bcaf16e91d3a05 P 50b46afc138846a184b2b6631eb6d4a1: Neither blocks nor log segments found. Creating new log.
I20260810 18:50:51.801844 1865 tablet_bootstrap.cc:492] T b9036e4ad8604efab1bcaf16e91d3a05 P 002006ca84bf445fa0bfc3dae398659a: Bootstrap starting.
I20260810 18:50:51.805454 1867 log.cc:824] T b9036e4ad8604efab1bcaf16e91d3a05 P 50b46afc138846a184b2b6631eb6d4a1: Log is configured to *not* fsync() on all Append() calls
I20260810 18:50:51.806064 1865 tablet_bootstrap.cc:654] T b9036e4ad8604efab1bcaf16e91d3a05 P 002006ca84bf445fa0bfc3dae398659a: Neither blocks nor log segments found. Creating new log.
I20260810 18:50:51.807559 1865 log.cc:824] T b9036e4ad8604efab1bcaf16e91d3a05 P 002006ca84bf445fa0bfc3dae398659a: Log is configured to *not* fsync() on all Append() calls
I20260810 18:50:51.810526 1867 tablet_bootstrap.cc:492] T b9036e4ad8604efab1bcaf16e91d3a05 P 50b46afc138846a184b2b6631eb6d4a1: No bootstrap required, opened a new log
I20260810 18:50:51.810803 1867 ts_tablet_manager.cc:1397] T b9036e4ad8604efab1bcaf16e91d3a05 P 50b46afc138846a184b2b6631eb6d4a1: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260810 18:50:51.815233 1867 raft_consensus.cc:348] T b9036e4ad8604efab1bcaf16e91d3a05 P 50b46afc138846a184b2b6631eb6d4a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "002006ca84bf445fa0bfc3dae398659a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36225 } } peers { permanent_uuid: "115c47de96be486f9f6e0554b6724f15" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 44051 } } peers { permanent_uuid: "50b46afc138846a184b2b6631eb6d4a1" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43585 } }
I20260810 18:50:51.816497 1867 raft_consensus.cc:374] T b9036e4ad8604efab1bcaf16e91d3a05 P 50b46afc138846a184b2b6631eb6d4a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:50:51.816645 1867 raft_consensus.cc:729] T b9036e4ad8604efab1bcaf16e91d3a05 P 50b46afc138846a184b2b6631eb6d4a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50b46afc138846a184b2b6631eb6d4a1, State: Initialized, Role: FOLLOWER
I20260810 18:50:51.817369 1867 consensus_queue.cc:260] T b9036e4ad8604efab1bcaf16e91d3a05 P 50b46afc138846a184b2b6631eb6d4a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "002006ca84bf445fa0bfc3dae398659a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36225 } } peers { permanent_uuid: "115c47de96be486f9f6e0554b6724f15" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 44051 } } peers { permanent_uuid: "50b46afc138846a184b2b6631eb6d4a1" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43585 } }
I20260810 18:50:51.819792 1865 tablet_bootstrap.cc:492] T b9036e4ad8604efab1bcaf16e91d3a05 P 002006ca84bf445fa0bfc3dae398659a: No bootstrap required, opened a new log
I20260810 18:50:51.820101 1865 ts_tablet_manager.cc:1397] T b9036e4ad8604efab1bcaf16e91d3a05 P 002006ca84bf445fa0bfc3dae398659a: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.000s
I20260810 18:50:51.825599 1864 tablet_bootstrap.cc:492] T b9036e4ad8604efab1bcaf16e91d3a05 P 115c47de96be486f9f6e0554b6724f15: Bootstrap starting.
I20260810 18:50:51.825989 1867 ts_tablet_manager.cc:1428] T b9036e4ad8604efab1bcaf16e91d3a05 P 50b46afc138846a184b2b6631eb6d4a1: Time spent starting tablet: real 0.015s user 0.006s sys 0.000s
I20260810 18:50:51.824503 1865 raft_consensus.cc:348] T b9036e4ad8604efab1bcaf16e91d3a05 P 002006ca84bf445fa0bfc3dae398659a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "002006ca84bf445fa0bfc3dae398659a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36225 } } peers { permanent_uuid: "115c47de96be486f9f6e0554b6724f15" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 44051 } } peers { permanent_uuid: "50b46afc138846a184b2b6631eb6d4a1" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43585 } }
I20260810 18:50:51.826288 1865 raft_consensus.cc:374] T b9036e4ad8604efab1bcaf16e91d3a05 P 002006ca84bf445fa0bfc3dae398659a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:50:51.826442 1865 raft_consensus.cc:729] T b9036e4ad8604efab1bcaf16e91d3a05 P 002006ca84bf445fa0bfc3dae398659a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 002006ca84bf445fa0bfc3dae398659a, State: Initialized, Role: FOLLOWER
I20260810 18:50:51.827060 1865 consensus_queue.cc:260] T b9036e4ad8604efab1bcaf16e91d3a05 P 002006ca84bf445fa0bfc3dae398659a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "002006ca84bf445fa0bfc3dae398659a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36225 } } peers { permanent_uuid: "115c47de96be486f9f6e0554b6724f15" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 44051 } } peers { permanent_uuid: "50b46afc138846a184b2b6631eb6d4a1" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43585 } }
I20260810 18:50:51.828219 1865 ts_tablet_manager.cc:1428] T b9036e4ad8604efab1bcaf16e91d3a05 P 002006ca84bf445fa0bfc3dae398659a: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260810 18:50:51.828679 1513 heartbeater.cc:503] Master 127.18.168.190:43801 was elected leader, sending a full tablet report...
I20260810 18:50:51.831856 1864 tablet_bootstrap.cc:654] T b9036e4ad8604efab1bcaf16e91d3a05 P 115c47de96be486f9f6e0554b6724f15: Neither blocks nor log segments found. Creating new log.
I20260810 18:50:51.833357 1669 heartbeater.cc:503] Master 127.18.168.190:43801 was elected leader, sending a full tablet report...
I20260810 18:50:51.835176 1864 log.cc:824] T b9036e4ad8604efab1bcaf16e91d3a05 P 115c47de96be486f9f6e0554b6724f15: Log is configured to *not* fsync() on all Append() calls
I20260810 18:50:51.849849 1864 tablet_bootstrap.cc:492] T b9036e4ad8604efab1bcaf16e91d3a05 P 115c47de96be486f9f6e0554b6724f15: No bootstrap required, opened a new log
I20260810 18:50:51.850298 1864 ts_tablet_manager.cc:1397] T b9036e4ad8604efab1bcaf16e91d3a05 P 115c47de96be486f9f6e0554b6724f15: Time spent bootstrapping tablet: real 0.025s user 0.006s sys 0.000s
I20260810 18:50:51.856914 1864 raft_consensus.cc:348] T b9036e4ad8604efab1bcaf16e91d3a05 P 115c47de96be486f9f6e0554b6724f15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "002006ca84bf445fa0bfc3dae398659a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36225 } } peers { permanent_uuid: "115c47de96be486f9f6e0554b6724f15" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 44051 } } peers { permanent_uuid: "50b46afc138846a184b2b6631eb6d4a1" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43585 } }
I20260810 18:50:51.868389 1872 raft_consensus.cc:482] T b9036e4ad8604efab1bcaf16e91d3a05 P 002006ca84bf445fa0bfc3dae398659a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:50:51.868629 1872 raft_consensus.cc:504] T b9036e4ad8604efab1bcaf16e91d3a05 P 002006ca84bf445fa0bfc3dae398659a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "002006ca84bf445fa0bfc3dae398659a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36225 } } peers { permanent_uuid: "115c47de96be486f9f6e0554b6724f15" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 44051 } } peers { permanent_uuid: "50b46afc138846a184b2b6631eb6d4a1" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43585 } }
I20260810 18:50:51.870003 1872 leader_election.cc:288] T b9036e4ad8604efab1bcaf16e91d3a05 P 002006ca84bf445fa0bfc3dae398659a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 115c47de96be486f9f6e0554b6724f15 (127.18.168.131:44051), 50b46afc138846a184b2b6631eb6d4a1 (127.18.168.129:43585)
I20260810 18:50:51.876775 1864 raft_consensus.cc:374] T b9036e4ad8604efab1bcaf16e91d3a05 P 115c47de96be486f9f6e0554b6724f15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:50:51.876961 1864 raft_consensus.cc:729] T b9036e4ad8604efab1bcaf16e91d3a05 P 115c47de96be486f9f6e0554b6724f15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c47de96be486f9f6e0554b6724f15, State: Initialized, Role: FOLLOWER
I20260810 18:50:51.877648 1864 consensus_queue.cc:260] T b9036e4ad8604efab1bcaf16e91d3a05 P 115c47de96be486f9f6e0554b6724f15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "002006ca84bf445fa0bfc3dae398659a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36225 } } peers { permanent_uuid: "115c47de96be486f9f6e0554b6724f15" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 44051 } } peers { permanent_uuid: "50b46afc138846a184b2b6631eb6d4a1" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43585 } }
I20260810 18:50:51.878618 1824 heartbeater.cc:503] Master 127.18.168.190:43801 was elected leader, sending a full tablet report...
I20260810 18:50:51.879240 1864 ts_tablet_manager.cc:1428] T b9036e4ad8604efab1bcaf16e91d3a05 P 115c47de96be486f9f6e0554b6724f15: Time spent starting tablet: real 0.029s user 0.003s sys 0.005s
W20260810 18:50:51.884238 1514 tablet.cc:2369] T b9036e4ad8604efab1bcaf16e91d3a05 P 50b46afc138846a184b2b6631eb6d4a1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:50:51.888561 1862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:51.871106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:39227 (local address 127.18.168.129:43585)
0810 18:50:51.871263 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:51.871268 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:51.881171 (+ 9903us) server_negotiation.cc:407] Connection header received
0810 18:50:51.884051 (+ 2880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:51.884055 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:51.884124 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:51.884191 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:50:51.884643 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:51.885428 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:51.887210 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:51.887814 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:51.887931 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:51.887990 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:51.888356 (+ 366us) server_negotiation.cc:299] Negotiation successful
0810 18:50:51.888434 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 18:50:51.893657 1875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:51.870894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:43585 (local address 127.18.168.130:39227)
0810 18:50:51.881069 (+ 10175us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:51.881084 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:50:51.881239 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:50:51.884315 (+ 3076us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:50:51.884317 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:50:51.884348 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:50:51.884521 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:51.884527 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:51.886037 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:51.886039 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:51.887087 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:51.887096 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:51.888056 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:51.888060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:51.888204 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:51.888229 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 18:50:51.892415 (+ 4186us) client_negotiation.cc:240] Negotiation successful
0810 18:50:51.892504 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10106,"thread_start_us":179,"threads_started":1}
I20260810 18:50:51.894637 1439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9036e4ad8604efab1bcaf16e91d3a05" candidate_uuid: "002006ca84bf445fa0bfc3dae398659a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50b46afc138846a184b2b6631eb6d4a1" is_pre_election: true
I20260810 18:50:51.894963 1439 raft_consensus.cc:2454] T b9036e4ad8604efab1bcaf16e91d3a05 P 50b46afc138846a184b2b6631eb6d4a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 002006ca84bf445fa0bfc3dae398659a in term 0.
I20260810 18:50:51.895608 1539 leader_election.cc:302] T b9036e4ad8604efab1bcaf16e91d3a05 P 002006ca84bf445fa0bfc3dae398659a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 002006ca84bf445fa0bfc3dae398659a, 50b46afc138846a184b2b6631eb6d4a1; no voters:
I20260810 18:50:51.895906 1872 raft_consensus.cc:2781] T b9036e4ad8604efab1bcaf16e91d3a05 P 002006ca84bf445fa0bfc3dae398659a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:50:51.895982 1872 raft_consensus.cc:482] T b9036e4ad8604efab1bcaf16e91d3a05 P 002006ca84bf445fa0bfc3dae398659a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:50:51.896023 1872 raft_consensus.cc:3037] T b9036e4ad8604efab1bcaf16e91d3a05 P 002006ca84bf445fa0bfc3dae398659a [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:50:51.897770 1872 raft_consensus.cc:504] T b9036e4ad8604efab1bcaf16e91d3a05 P 002006ca84bf445fa0bfc3dae398659a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "002006ca84bf445fa0bfc3dae398659a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36225 } } peers { permanent_uuid: "115c47de96be486f9f6e0554b6724f15" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 44051 } } peers { permanent_uuid: "50b46afc138846a184b2b6631eb6d4a1" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43585 } }
I20260810 18:50:51.898136 1872 leader_election.cc:288] T b9036e4ad8604efab1bcaf16e91d3a05 P 002006ca84bf445fa0bfc3dae398659a [CANDIDATE]: Term 1 election: Requested vote from peers 115c47de96be486f9f6e0554b6724f15 (127.18.168.131:44051), 50b46afc138846a184b2b6631eb6d4a1 (127.18.168.129:43585)
I20260810 18:50:51.898787 1439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9036e4ad8604efab1bcaf16e91d3a05" candidate_uuid: "002006ca84bf445fa0bfc3dae398659a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50b46afc138846a184b2b6631eb6d4a1"
I20260810 18:50:51.898908 1439 raft_consensus.cc:3037] T b9036e4ad8604efab1bcaf16e91d3a05 P 50b46afc138846a184b2b6631eb6d4a1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:50:51.899014 1860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:51.871471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:40041 (local address 127.18.168.131:44051)
0810 18:50:51.871604 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:51.871608 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:51.879702 (+ 8094us) server_negotiation.cc:407] Connection header received
0810 18:50:51.880099 (+ 397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:51.880105 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:51.880162 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:51.880213 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:50:51.881357 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:51.882167 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:51.885422 (+ 3255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:51.885876 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:51.885949 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:51.885990 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:51.898791 (+ 12801us) server_negotiation.cc:299] Negotiation successful
0810 18:50:51.898896 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 18:50:51.899410 1874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:51.870427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.131:44051 (local address 127.18.168.130:40041)
0810 18:50:51.879556 (+ 9129us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:51.879574 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:50:51.879963 (+ 389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:50:51.880368 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:50:51.880374 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:50:51.880398 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:50:51.880795 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:51.880807 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:51.882307 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:51.882313 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:51.883833 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:51.883844 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:51.887286 (+ 3442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:51.887289 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:51.887468 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:51.887494 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 18:50:51.899145 (+ 11651us) client_negotiation.cc:240] Negotiation successful
0810 18:50:51.899232 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9051,"thread_start_us":192,"threads_started":1}
I20260810 18:50:51.901791 1773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9036e4ad8604efab1bcaf16e91d3a05" candidate_uuid: "002006ca84bf445fa0bfc3dae398659a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c47de96be486f9f6e0554b6724f15" is_pre_election: true
I20260810 18:50:51.902102 1773 raft_consensus.cc:2454] T b9036e4ad8604efab1bcaf16e91d3a05 P 115c47de96be486f9f6e0554b6724f15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 002006ca84bf445fa0bfc3dae398659a in term 0.
I20260810 18:50:51.902565 1771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9036e4ad8604efab1bcaf16e91d3a05" candidate_uuid: "002006ca84bf445fa0bfc3dae398659a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c47de96be486f9f6e0554b6724f15"
I20260810 18:50:51.902673 1771 raft_consensus.cc:3037] T b9036e4ad8604efab1bcaf16e91d3a05 P 115c47de96be486f9f6e0554b6724f15 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:50:51.905769 1771 raft_consensus.cc:2454] T b9036e4ad8604efab1bcaf16e91d3a05 P 115c47de96be486f9f6e0554b6724f15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 002006ca84bf445fa0bfc3dae398659a in term 1.
I20260810 18:50:51.906216 1536 leader_election.cc:302] T b9036e4ad8604efab1bcaf16e91d3a05 P 002006ca84bf445fa0bfc3dae398659a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 002006ca84bf445fa0bfc3dae398659a, 115c47de96be486f9f6e0554b6724f15; no voters:
I20260810 18:50:51.906473 1872 raft_consensus.cc:2781] T b9036e4ad8604efab1bcaf16e91d3a05 P 002006ca84bf445fa0bfc3dae398659a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:50:51.906805 1872 raft_consensus.cc:686] T b9036e4ad8604efab1bcaf16e91d3a05 P 002006ca84bf445fa0bfc3dae398659a [term 1 LEADER]: Becoming Leader. State: Replica: 002006ca84bf445fa0bfc3dae398659a, State: Running, Role: LEADER
I20260810 18:50:51.907200 1872 consensus_queue.cc:237] T b9036e4ad8604efab1bcaf16e91d3a05 P 002006ca84bf445fa0bfc3dae398659a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "002006ca84bf445fa0bfc3dae398659a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36225 } } peers { permanent_uuid: "115c47de96be486f9f6e0554b6724f15" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 44051 } } peers { permanent_uuid: "50b46afc138846a184b2b6631eb6d4a1" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43585 } }
I20260810 18:50:51.907598 1439 raft_consensus.cc:2454] T b9036e4ad8604efab1bcaf16e91d3a05 P 50b46afc138846a184b2b6631eb6d4a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 002006ca84bf445fa0bfc3dae398659a in term 1.
W20260810 18:50:51.908664 1670 tablet.cc:2369] T b9036e4ad8604efab1bcaf16e91d3a05 P 002006ca84bf445fa0bfc3dae398659a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:50:51.912097 1825 tablet.cc:2369] T b9036e4ad8604efab1bcaf16e91d3a05 P 115c47de96be486f9f6e0554b6724f15: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:50:51.913383 1285 catalog_manager.cc:5495] T b9036e4ad8604efab1bcaf16e91d3a05 P 002006ca84bf445fa0bfc3dae398659a reported cstate change: term changed from 0 to 1, leader changed from <none> to 002006ca84bf445fa0bfc3dae398659a (127.18.168.130). New cstate: current_term: 1 leader_uuid: "002006ca84bf445fa0bfc3dae398659a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "002006ca84bf445fa0bfc3dae398659a" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 36225 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "115c47de96be486f9f6e0554b6724f15" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 44051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50b46afc138846a184b2b6631eb6d4a1" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43585 } health_report { overall_health: UNKNOWN } } }
I20260810 18:50:51.943616 1858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:51.938706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51418 (local address 127.18.168.130:36225)
0810 18:50:51.938815 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:51.938819 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:51.938831 (+ 12us) server_negotiation.cc:407] Connection header received
0810 18:50:51.938909 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:51.938912 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:51.938974 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:51.939076 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:50:51.939534 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:51.940280 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:51.941554 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:51.941863 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:51.941949 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:51.942344 (+ 395us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:50:51.942891 (+ 547us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:50:51.943006 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:51.943381 (+ 375us) server_negotiation.cc:299] Negotiation successful
0810 18:50:51.943451 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 18:50:51.951646 1439 raft_consensus.cc:1261] T b9036e4ad8604efab1bcaf16e91d3a05 P 50b46afc138846a184b2b6631eb6d4a1 [term 1 FOLLOWER]: Refusing update from remote peer 002006ca84bf445fa0bfc3dae398659a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:50:51.952536 1877 consensus_queue.cc:1035] T b9036e4ad8604efab1bcaf16e91d3a05 P 002006ca84bf445fa0bfc3dae398659a [LEADER]: Connected to new peer: Peer: permanent_uuid: "50b46afc138846a184b2b6631eb6d4a1" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:50:51.959666 1771 raft_consensus.cc:1261] T b9036e4ad8604efab1bcaf16e91d3a05 P 115c47de96be486f9f6e0554b6724f15 [term 1 FOLLOWER]: Refusing update from remote peer 002006ca84bf445fa0bfc3dae398659a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:50:51.960381 1877 consensus_queue.cc:1035] T b9036e4ad8604efab1bcaf16e91d3a05 P 002006ca84bf445fa0bfc3dae398659a [LEADER]: Connected to new peer: Peer: permanent_uuid: "115c47de96be486f9f6e0554b6724f15" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 44051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:50:51.974252 1880 mvcc.cc:205] Tried to move back new op lower bound from 7317044641584783360 to 7317044641423167488. Current Snapshot: MvccSnapshot[applied={T|T < 7317044641584783360}]
I20260810 18:50:51.980773 1884 mvcc.cc:205] Tried to move back new op lower bound from 7317044641584783360 to 7317044641423167488. Current Snapshot: MvccSnapshot[applied={T|T < 7317044641584783360}]
I20260810 18:50:51.982369 1882 mvcc.cc:205] Tried to move back new op lower bound from 7317044641584783360 to 7317044641423167488. Current Snapshot: MvccSnapshot[applied={T|T < 7317044641584783360}]
I20260810 18:50:51.991052 1284 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59970:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:50:51.991227 1284 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59970:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:50:51.994127 1284 catalog_manager.cc:5782] T 00000000000000000000000000000000 P db5a963396c94f9ca5b846feadc654f7: Sending DeleteTablet for 3 replicas of tablet b9036e4ad8604efab1bcaf16e91d3a05
I20260810 18:50:51.995513 1594 tablet_service.cc:1514] Processing DeleteTablet for tablet b9036e4ad8604efab1bcaf16e91d3a05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:50:51 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:51410
I20260810 18:50:51.996240 1437 tablet_service.cc:1514] Processing DeleteTablet for tablet b9036e4ad8604efab1bcaf16e91d3a05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:50:51 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:51022
I20260810 18:50:51.997499 1750 tablet_service.cc:1514] Processing DeleteTablet for tablet b9036e4ad8604efab1bcaf16e91d3a05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:50:51 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:50256
I20260810 18:50:51.998905 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 1328
I20260810 18:50:52.005131 1895 tablet_replica.cc:330] T b9036e4ad8604efab1bcaf16e91d3a05 P 115c47de96be486f9f6e0554b6724f15: stopping tablet replica
I20260810 18:50:52.005538 1893 tablet_replica.cc:330] T b9036e4ad8604efab1bcaf16e91d3a05 P 002006ca84bf445fa0bfc3dae398659a: stopping tablet replica
I20260810 18:50:52.012481 1895 raft_consensus.cc:2229] T b9036e4ad8604efab1bcaf16e91d3a05 P 115c47de96be486f9f6e0554b6724f15 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:50:52.012909 1893 raft_consensus.cc:2229] T b9036e4ad8604efab1bcaf16e91d3a05 P 002006ca84bf445fa0bfc3dae398659a [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:50:52.013417 1893 raft_consensus.cc:2258] T b9036e4ad8604efab1bcaf16e91d3a05 P 002006ca84bf445fa0bfc3dae398659a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:50:52.013830 1895 raft_consensus.cc:2258] T b9036e4ad8604efab1bcaf16e91d3a05 P 115c47de96be486f9f6e0554b6724f15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:50:52.015669 1893 ts_tablet_manager.cc:1905] T b9036e4ad8604efab1bcaf16e91d3a05 P 002006ca84bf445fa0bfc3dae398659a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:50:52.016613 1895 ts_tablet_manager.cc:1905] T b9036e4ad8604efab1bcaf16e91d3a05 P 115c47de96be486f9f6e0554b6724f15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:50:52.019476 1895 ts_tablet_manager.cc:1918] T b9036e4ad8604efab1bcaf16e91d3a05 P 115c47de96be486f9f6e0554b6724f15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:50:52.030920 1893 ts_tablet_manager.cc:1918] T b9036e4ad8604efab1bcaf16e91d3a05 P 002006ca84bf445fa0bfc3dae398659a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:50:52.031134 1895 log.cc:1196] T b9036e4ad8604efab1bcaf16e91d3a05 P 115c47de96be486f9f6e0554b6724f15: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-2/wal/wals/b9036e4ad8604efab1bcaf16e91d3a05
I20260810 18:50:52.031647 1895 ts_tablet_manager.cc:1939] T b9036e4ad8604efab1bcaf16e91d3a05 P 115c47de96be486f9f6e0554b6724f15: Deleting consensus metadata
I20260810 18:50:52.033282 1267 catalog_manager.cc:4841] TS 115c47de96be486f9f6e0554b6724f15 (127.18.168.131:44051): tablet b9036e4ad8604efab1bcaf16e91d3a05 (table test-table [id=545e80d005e349c28170569925e976d2]) successfully deleted
I20260810 18:50:52.033640 1893 log.cc:1196] T b9036e4ad8604efab1bcaf16e91d3a05 P 002006ca84bf445fa0bfc3dae398659a: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387687857264-19106-0/minicluster-data/ts-1/wal/wals/b9036e4ad8604efab1bcaf16e91d3a05
I20260810 18:50:52.034065 1893 ts_tablet_manager.cc:1939] T b9036e4ad8604efab1bcaf16e91d3a05 P 002006ca84bf445fa0bfc3dae398659a: Deleting consensus metadata
W20260810 18:50:52.034360 1269 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.168.129:43585 (error 108)
I20260810 18:50:52.036132 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 1518
I20260810 18:50:52.058000 1859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:52.057618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:43585 (local address 127.0.0.1:51026)
0810 18:50:52.057781 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:52.057887 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.129:43585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260810 18:50:52.058245 1269 catalog_manager.cc:4584] TS 50b46afc138846a184b2b6631eb6d4a1 (127.18.168.129:43585): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b9036e4ad8604efab1bcaf16e91d3a05: Network error: Client connection negotiation failed: client connection to 127.18.168.129:43585: connect: Connection refused (error 111)
W20260810 18:50:52.073065 1859 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:50:52.072220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:36225 (local address 127.0.0.1:51428)
0810 18:50:52.072425 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:52.072438 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:50:52.072524 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:50:52.072905 (+ 381us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.130:36225: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":121}
I20260810 18:50:52.073122 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 1673
I20260810 18:50:52.082702 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 1248
I20260810 18:50:52.093613 19106 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 18:50:52.438568 19106 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 18:50:52.438674 19106 mini_ranger.cc:66] Stopping Ranger...
I20260810 18:50:52.794003 19106 mini_ranger.cc:68] Stopped Ranger
2026-08-10 18:50:52.794 UTC [32654] LOG: received smart shutdown request
2026-08-10 18:50:52.798 UTC [32654] LOG: background worker "logical replication launcher" (PID 32664) exited with exit code 1
2026-08-10 18:50:52.798 UTC [32659] LOG: shutting down
2026-08-10 18:50:52.971 UTC [32654] LOG: database system is shut down
2026-08-10T18:50:52Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (48887 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:50:53 dist-test-slave-cbsf krb5kdc[1908](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:50:53 dist-test-slave-cbsf krb5kdc[1908](info): set up 2 sockets
Aug 10 18:50:53 dist-test-slave-cbsf krb5kdc[1908](info): commencing operation
krb5kdc: starting...
W20260810 18:50:54.321153 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:50:54 dist-test-slave-cbsf krb5kdc[1908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:50:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:50:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.168.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
I20260810 18:50:54.443482 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:34471
--webserver_interface=127.18.168.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:44265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:34471,127.18.168.189:33095,127.18.168.188:44755
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:50:54.555819 1923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:50:54.556083 1923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:50:54.556165 1923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:50:54.560055 1923 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:50:54.560135 1923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:50:54.560170 1923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:50:54.560222 1923 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:50:54.560257 1923 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:50:54.565310 1923 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:44265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:34471,127.18.168.189:33095,127.18.168.188:44755
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:34471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1923
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:50:54.566303 1923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:50:54.567277 1923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:50:54.575762 1930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:50:54.575762 1934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:50:54.575966 1923 server_base.cc:1034] running on GCE node
W20260810 18:50:54.575834 1928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:50:54.576608 1923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:50:54.577702 1923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:50:54.578894 1923 hybrid_clock.cc:648] HybridClock initialized: now 1786387854578873 us; error 33 us; skew 500 ppm
Aug 10 18:50:54 dist-test-slave-cbsf krb5kdc[1908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387854, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:50:54.581641 1923 init.cc:377] Logged in from keytab as kudu/127.18.168.190@KRBTEST.COM (short username kudu)
I20260810 18:50:54.582504 1923 webserver.cc:460] Webserver started at http://127.18.168.190:38787/ using document root <none> and password file <none>
I20260810 18:50:54.583001 1923 fs_manager.cc:359] Metadata directory not provided
I20260810 18:50:54.583062 1923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:50:54.583230 1923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:50:54.585172 1923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "1f8de7a0e6964ff586e98164489935d4"
format_stamp: "Formatted at 2026-08-10 18:50:54 on dist-test-slave-cbsf"
server_key: "726b747f7d35c995d5c6d197769336dd"
server_key_iv: "ba725ddebf80cb7bfa5f72d6ef6a057a"
server_key_version: "encryptionkey@0"
I20260810 18:50:54.585647 1923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "1f8de7a0e6964ff586e98164489935d4"
format_stamp: "Formatted at 2026-08-10 18:50:54 on dist-test-slave-cbsf"
server_key: "726b747f7d35c995d5c6d197769336dd"
server_key_iv: "ba725ddebf80cb7bfa5f72d6ef6a057a"
server_key_version: "encryptionkey@0"
I20260810 18:50:54.589160 1923 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:50:54.591363 1942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:54.592561 1923 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:50:54.592664 1923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "1f8de7a0e6964ff586e98164489935d4"
format_stamp: "Formatted at 2026-08-10 18:50:54 on dist-test-slave-cbsf"
server_key: "726b747f7d35c995d5c6d197769336dd"
server_key_iv: "ba725ddebf80cb7bfa5f72d6ef6a057a"
server_key_version: "encryptionkey@0"
I20260810 18:50:54.592823 1923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:50:54.606706 1923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:50:54.609298 1923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:50:54.609439 1923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:50:54.617650 1923 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:34471
I20260810 18:50:54.617911 2001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:34471 every 8 connection(s)
I20260810 18:50:54.618628 1923 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:50:54.619638 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 1923
I20260810 18:50:54.619765 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:50:54.620069 19106 external_mini_cluster.cc:1428] Setting key 58415e55571fe3bfffecfbbd5cb91cf7
I20260810 18:50:54.622061 2004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:50:54.626449 2004 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.168.190" port: 34471 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.168.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.189@KRBTEST.COM" created.
Aug 10 18:50:54 dist-test-slave-cbsf krb5kdc[1908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387854, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:50:54.642233 2009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:54.628676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:34471 (local address 127.0.0.1:41438)
0810 18:50:54.629295 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:54.629329 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 18:50:54.630172 (+ 843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:50:54.630832 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:50:54.630837 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:50:54.631233 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:50:54.631465 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:54.631477 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:54.632797 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:54.632800 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:54.633533 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:54.633557 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:54.634534 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:54.634542 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:54.634741 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:54.635358 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:50:54.635384 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:50:54.637192 (+ 1808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:50:54.639221 (+ 2029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:54.639232 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:54.639241 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:54.639464 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:54.639812 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:54.639815 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:54.639817 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:54.639928 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:54.640488 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:50:54.640493 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:50:54.640649 (+ 156us) client_negotiation.cc:769] Sending connection context
0810 18:50:54.640865 (+ 216us) client_negotiation.cc:240] Negotiation successful
0810 18:50:54.641276 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"mutex_wait_us":129,"thread_start_us":142,"threads_started":1}
I20260810 18:50:54.642233 2008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:54.628675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41438 (local address 127.18.168.190:34471)
0810 18:50:54.629105 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:54.629119 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:54.629433 (+ 314us) server_negotiation.cc:407] Connection header received
0810 18:50:54.630482 (+ 1049us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:54.630486 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:54.630653 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:54.630832 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:50:54.631623 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:54.632595 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:54.633678 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:54.634325 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:54.634518 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:54.637366 (+ 2848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:50:54.637372 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:50:54.637391 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:50:54.637411 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:50:54.639108 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:54.639606 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:54.639615 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:54.639616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:54.639713 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:54.640032 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:54.640035 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:54.640037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:54.640366 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:50:54.640503 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:54.640948 (+ 445us) server_negotiation.cc:299] Negotiation successful
0810 18:50:54.641226 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":248,"thread_start_us":128,"threads_started":1}
I20260810 18:50:54.644634 2004 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33095 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:50:54.645380 2009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:54.645145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.189:33095 (local address 127.0.0.1:40314)
0810 18:50:54.645258 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:54.645321 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.189:33095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 18:50:54.645586 1947 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.168.189:33095: connect: Connection refused (error 111)
I20260810 18:50:54.646425 2009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:54.646248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.189:33095 (local address 127.0.0.1:40320)
0810 18:50:54.646338 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:54.646378 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.189:33095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260810 18:50:54.646584 2004 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.168.189:33095: Network error: Client connection negotiation failed: client connection to 127.18.168.189:33095: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.189.keytab.
Entry for principal kudu/127.18.168.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.189.keytab.
I20260810 18:50:54.651175 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.189:33095
--webserver_interface=127.18.168.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.189.keytab
--principal=kudu/127.18.168.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:44265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:34471,127.18.168.189:33095,127.18.168.188:44755
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:50:54.695856 2004 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33095 } attempt: 1
I20260810 18:50:54.696900 2009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:54.696494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.189:33095 (local address 127.0.0.1:40332)
0810 18:50:54.696729 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:54.696823 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.189:33095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260810 18:50:54.697785 2009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:54.697576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.189:33095 (local address 127.0.0.1:40336)
0810 18:50:54.697684 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:54.697729 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.189:33095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260810 18:50:54.697954 2004 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.168.189:33095: Network error: Client connection negotiation failed: client connection to 127.18.168.189:33095: connect: Connection refused (error 111)
W20260810 18:50:54.761930 2012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:50:54.762162 2012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:50:54.762199 2012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:50:54.765944 2012 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:50:54.765992 2012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:50:54.766010 2012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:50:54.766026 2012 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:50:54.766040 2012 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:50:54.766163 2004 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33095 } attempt: 2
I20260810 18:50:54.766965 2009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:54.766670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.189:33095 (local address 127.0.0.1:40342)
0810 18:50:54.766819 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:54.766889 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.189:33095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260810 18:50:54.767712 2009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:54.767515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.189:33095 (local address 127.0.0.1:40344)
0810 18:50:54.767615 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:54.767660 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.189:33095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 18:50:54.767921 2004 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.168.189:33095: Network error: Client connection negotiation failed: client connection to 127.18.168.189:33095: connect: Connection refused (error 111)
I20260810 18:50:54.771695 2012 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:44265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:34471,127.18.168.189:33095,127.18.168.188:44755
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.189.keytab
--principal=kudu/127.18.168.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.189:33095
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.168.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:50:54.772774 2012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:50:54.773784 2012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:50:54.782313 2018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:50:54.782449 2012 server_base.cc:1034] running on GCE node
W20260810 18:50:54.782348 2017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:50:54.782313 2021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:50:54.783052 2012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:50:54.784137 2012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:50:54.785328 2012 hybrid_clock.cc:648] HybridClock initialized: now 1786387854785298 us; error 39 us; skew 500 ppm
Aug 10 18:50:54 dist-test-slave-cbsf krb5kdc[1908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387854, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:50:54.788137 2012 init.cc:377] Logged in from keytab as kudu/127.18.168.189@KRBTEST.COM (short username kudu)
I20260810 18:50:54.789085 2012 webserver.cc:460] Webserver started at http://127.18.168.189:43227/ using document root <none> and password file <none>
I20260810 18:50:54.789624 2012 fs_manager.cc:359] Metadata directory not provided
I20260810 18:50:54.789728 2012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:50:54.789963 2012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:50:54.791949 2012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-1/data/instance:
uuid: "3ff6e91786b24e6a96fafa79313db149"
format_stamp: "Formatted at 2026-08-10 18:50:54 on dist-test-slave-cbsf"
server_key: "6ab6bfa2b23e39a297db0e0aad92ee3a"
server_key_iv: "36903217fa40273c48df3f57293b8d8f"
server_key_version: "encryptionkey@0"
I20260810 18:50:54.792564 2012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-1/wal/instance:
uuid: "3ff6e91786b24e6a96fafa79313db149"
format_stamp: "Formatted at 2026-08-10 18:50:54 on dist-test-slave-cbsf"
server_key: "6ab6bfa2b23e39a297db0e0aad92ee3a"
server_key_iv: "36903217fa40273c48df3f57293b8d8f"
server_key_version: "encryptionkey@0"
I20260810 18:50:54.796337 2012 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:50:54.798909 2028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:54.800115 2012 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:50:54.800263 2012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-1/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-1/wal
uuid: "3ff6e91786b24e6a96fafa79313db149"
format_stamp: "Formatted at 2026-08-10 18:50:54 on dist-test-slave-cbsf"
server_key: "6ab6bfa2b23e39a297db0e0aad92ee3a"
server_key_iv: "36903217fa40273c48df3f57293b8d8f"
server_key_version: "encryptionkey@0"
I20260810 18:50:54.800374 2012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:50:54.816500 2012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:50:54.820050 2012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:50:54.820279 2012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:50:54.828236 2012 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.189:33095
I20260810 18:50:54.828553 2095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.189:33095 every 8 connection(s)
I20260810 18:50:54.829294 2012 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-1/data/info.pb
I20260810 18:50:54.832082 2096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:50:54.835827 2096 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.168.190" port: 34471 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:50:54.837289 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 2012
I20260810 18:50:54.837435 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-1/wal/instance
I20260810 18:50:54.837774 19106 external_mini_cluster.cc:1428] Setting key 409c958898141388bdf1242087b8c410
Aug 10 18:50:54 dist-test-slave-cbsf krb5kdc[1908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387854, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.189@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.168.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.188@KRBTEST.COM" created.
I20260810 18:50:54.849973 2008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:54.837857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41440 (local address 127.18.168.190:34471)
0810 18:50:54.838004 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:54.838010 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:54.838903 (+ 893us) server_negotiation.cc:407] Connection header received
0810 18:50:54.839867 (+ 964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:54.839871 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:54.839928 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:54.840043 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:50:54.841356 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:54.841966 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:54.843373 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:54.843634 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:54.843756 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:54.846230 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:50:54.846233 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:50:54.846235 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:50:54.846258 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:50:54.847649 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:54.848181 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:54.848185 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:54.848187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:54.848300 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:54.848705 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:54.848711 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:54.848713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:54.848936 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:50:54.849064 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:54.849730 (+ 666us) server_negotiation.cc:299] Negotiation successful
0810 18:50:54.849839 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 18:50:54.851197 2100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:54.838224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:34471 (local address 127.0.0.1:41440)
0810 18:50:54.838728 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:54.838782 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 18:50:54.839650 (+ 868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:50:54.840236 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:50:54.840257 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:50:54.840608 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:50:54.841187 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:54.841201 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:54.842099 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:54.842102 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:54.843219 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:54.843229 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:54.843799 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:54.843804 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:54.843977 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:54.844573 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:50:54.844589 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:50:54.846005 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:50:54.847781 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:54.847786 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:54.847799 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:54.848052 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:54.848398 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:54.848402 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:54.848404 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:54.848593 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:54.849064 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:50:54.849070 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:50:54.849346 (+ 276us) client_negotiation.cc:769] Sending connection context
0810 18:50:54.849637 (+ 291us) client_negotiation.cc:240] Negotiation successful
0810 18:50:54.850007 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":139,"threads_started":1}
I20260810 18:50:54.852732 2096 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33095 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:50:54.859102 2004 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33095 } attempt: 3
Aug 10 18:50:54 dist-test-slave-cbsf krb5kdc[1908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387854, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.189@KRBTEST.COM for kudu/127.18.168.189@KRBTEST.COM
Aug 10 18:50:54 dist-test-slave-cbsf krb5kdc[1908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387854, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for kudu/127.18.168.189@KRBTEST.COM
I20260810 18:50:54.865509 2100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:54.853299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.189:33095 (local address 127.0.0.1:40346)
0810 18:50:54.853498 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:54.853511 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:50:54.853574 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:50:54.854181 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:50:54.854187 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:50:54.854387 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:50:54.854543 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:54.854553 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:54.855412 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:54.855416 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:54.856267 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:54.856279 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:54.856914 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:54.856921 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:54.857052 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:54.857506 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:50:54.857534 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:50:54.861397 (+ 3863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:50:54.863567 (+ 2170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:54.863571 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:54.863573 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:54.863901 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:54.864261 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:54.864265 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:54.864267 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:54.864371 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:54.864977 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:50:54.864983 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:50:54.865096 (+ 113us) client_negotiation.cc:769] Sending connection context
0810 18:50:54.865212 (+ 116us) client_negotiation.cc:240] Negotiation successful
0810 18:50:54.865341 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127}
I20260810 18:50:54.865931 2103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:54.853485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40346 (local address 127.18.168.189:33095)
0810 18:50:54.853851 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:54.853860 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:54.853876 (+ 16us) server_negotiation.cc:407] Connection header received
0810 18:50:54.853906 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:54.853910 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:54.854044 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:54.854143 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:50:54.854670 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:54.855300 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:54.856385 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:54.856800 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:54.856878 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:54.861567 (+ 4689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:50:54.861578 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:50:54.861592 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:50:54.861620 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:50:54.863405 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:54.864023 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:54.864029 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:54.864031 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:54.864143 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:54.864494 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:54.864497 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:54.864499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:54.864833 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:50:54.864993 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:54.865313 (+ 320us) server_negotiation.cc:299] Negotiation successful
0810 18:50:54.865478 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":151,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.188.keytab.
Entry for principal kudu/127.18.168.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.188.keytab.
I20260810 18:50:54.867877 2096 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.168.188" port: 44755 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:50:54.868615 2100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:54.868335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:44755 (local address 127.0.0.1:49084)
0810 18:50:54.868461 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:54.868547 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.188:44755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260810 18:50:54.868863 2032 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.168.188:44755: connect: Connection refused (error 111)
I20260810 18:50:54.869022 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.188:44755
--webserver_interface=127.18.168.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.188.keytab
--principal=kudu/127.18.168.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:44265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:34471,127.18.168.189:33095,127.18.168.188:44755
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:50:54.869812 2104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:54.859553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40358 (local address 127.18.168.189:33095)
0810 18:50:54.860023 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:54.860034 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:54.860053 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:50:54.860082 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:54.860084 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:54.860117 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:54.860234 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:50:54.860778 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:54.861583 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:54.863076 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:54.863480 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:54.863580 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:54.865754 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:50:54.865757 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:50:54.865760 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:50:54.865784 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:50:54.867339 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:54.867772 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:54.867777 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:54.867780 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:54.867868 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:54.868222 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:54.868228 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:54.868232 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:54.868457 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:50:54.868555 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:54.869600 (+ 1045us) server_negotiation.cc:299] Negotiation successful
0810 18:50:54.869693 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":99,"threads_started":1}
I20260810 18:50:54.869962 2009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:54.859457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.189:33095 (local address 127.0.0.1:40358)
0810 18:50:54.859577 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:54.859590 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:50:54.859687 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:50:54.860249 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:50:54.860253 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:50:54.860452 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:50:54.860612 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:54.860624 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:54.861707 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:54.861710 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:54.862949 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:54.862960 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:54.863592 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:54.863596 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:54.863718 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:54.864170 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:50:54.864185 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:50:54.865619 (+ 1434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:50:54.867466 (+ 1847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:54.867469 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:54.867471 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:54.867665 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:54.867992 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:54.867998 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:54.868003 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:54.868102 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:54.868555 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:50:54.868558 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:50:54.868680 (+ 122us) client_negotiation.cc:769] Sending connection context
0810 18:50:54.869541 (+ 861us) client_negotiation.cc:240] Negotiation successful
0810 18:50:54.869718 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260810 18:50:54.870738 2004 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.168.188" port: 44755 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:50:54.870925 2100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:54.870626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:44755 (local address 127.0.0.1:49096)
0810 18:50:54.870807 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:54.870855 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.188:44755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
W20260810 18:50:54.871100 2096 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.168.188:44755: Network error: Client connection negotiation failed: client connection to 127.18.168.188:44755: connect: Connection refused (error 111)
I20260810 18:50:54.871438 2009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:54.871193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:44755 (local address 127.0.0.1:49112)
0810 18:50:54.871305 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:54.871372 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.188:44755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 18:50:54.872182 2009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:54.871918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:44755 (local address 127.0.0.1:49124)
0810 18:50:54.872084 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:54.872136 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.188:44755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260810 18:50:54.872325 2004 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.168.188:44755: Network error: Client connection negotiation failed: client connection to 127.18.168.188:44755: connect: Connection refused (error 111)
I20260810 18:50:54.903496 2004 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.168.188" port: 44755 } attempt: 1
I20260810 18:50:54.904465 2009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:54.904097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:44755 (local address 127.0.0.1:49136)
0810 18:50:54.904250 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:54.904359 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.188:44755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260810 18:50:54.905350 2009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:54.905147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:44755 (local address 127.0.0.1:49138)
0810 18:50:54.905251 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:54.905296 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.188:44755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260810 18:50:54.905507 2004 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.168.188:44755: Network error: Client connection negotiation failed: client connection to 127.18.168.188:44755: connect: Connection refused (error 111)
I20260810 18:50:54.920408 2096 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.168.188" port: 44755 } attempt: 1
I20260810 18:50:54.921267 2100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:54.920979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:44755 (local address 127.0.0.1:49144)
0810 18:50:54.921130 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:54.921191 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.188:44755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260810 18:50:54.921976 2100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:54.921766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:44755 (local address 127.0.0.1:49146)
0810 18:50:54.921875 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:54.921926 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.188:44755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260810 18:50:54.922101 2096 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.168.188:44755: Network error: Client connection negotiation failed: client connection to 127.18.168.188:44755: connect: Connection refused (error 111)
I20260810 18:50:54.980665 2004 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.168.188" port: 44755 } attempt: 2
I20260810 18:50:54.981573 2009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:54.981288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:44755 (local address 127.0.0.1:49160)
0810 18:50:54.981438 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:54.981500 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.188:44755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260810 18:50:54.982283 2009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:54.982063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:44755 (local address 127.0.0.1:49162)
0810 18:50:54.982185 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:54.982232 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.188:44755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260810 18:50:54.982407 2004 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.168.188:44755: Network error: Client connection negotiation failed: client connection to 127.18.168.188:44755: connect: Connection refused (error 111)
W20260810 18:50:54.983400 2106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:50:54.983670 2106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:50:54.983748 2106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:50:54.987820 2106 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:50:54.987895 2106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:50:54.987941 2106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:50:54.988000 2106 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:50:54.988037 2106 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:50:54.990250 2096 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.168.188" port: 44755 } attempt: 2
I20260810 18:50:54.990917 2100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:54.990667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:44755 (local address 127.0.0.1:49164)
0810 18:50:54.990790 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:54.990851 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.188:44755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260810 18:50:54.991736 2100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:54.991455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:44755 (local address 127.0.0.1:49174)
0810 18:50:54.991572 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:54.991668 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.188:44755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260810 18:50:54.991915 2096 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.168.188:44755: Network error: Client connection negotiation failed: client connection to 127.18.168.188:44755: connect: Connection refused (error 111)
I20260810 18:50:54.995656 2106 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:44265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:34471,127.18.168.189:33095,127.18.168.188:44755
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.188.keytab
--principal=kudu/127.18.168.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.188:44755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.168.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2106
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:50:54.997069 2106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:50:54.998517 2106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:50:55.007010 2112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:50:55.007021 2111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:50:55.007030 2115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:50:55.007228 2106 server_base.cc:1034] running on GCE node
I20260810 18:50:55.007841 2106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:50:55.008965 2106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:50:55.010130 2106 hybrid_clock.cc:648] HybridClock initialized: now 1786387855010120 us; error 28 us; skew 500 ppm
Aug 10 18:50:55 dist-test-slave-cbsf krb5kdc[1908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387855, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:50:55.013025 2106 init.cc:377] Logged in from keytab as kudu/127.18.168.188@KRBTEST.COM (short username kudu)
I20260810 18:50:55.013906 2106 webserver.cc:460] Webserver started at http://127.18.168.188:44259/ using document root <none> and password file <none>
I20260810 18:50:55.014513 2106 fs_manager.cc:359] Metadata directory not provided
I20260810 18:50:55.014576 2106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:50:55.014735 2106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:50:55.016834 2106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-2/data/instance:
uuid: "8db490d070364857829b9c3b8d5c4bc3"
format_stamp: "Formatted at 2026-08-10 18:50:55 on dist-test-slave-cbsf"
server_key: "a4ae893c779124d929861583727995d9"
server_key_iv: "7ac53e589eedef6f83dd117ec797b15c"
server_key_version: "encryptionkey@0"
I20260810 18:50:55.017369 2106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-2/wal/instance:
uuid: "8db490d070364857829b9c3b8d5c4bc3"
format_stamp: "Formatted at 2026-08-10 18:50:55 on dist-test-slave-cbsf"
server_key: "a4ae893c779124d929861583727995d9"
server_key_iv: "7ac53e589eedef6f83dd117ec797b15c"
server_key_version: "encryptionkey@0"
I20260810 18:50:55.020970 2106 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:50:55.023203 2122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:55.024396 2106 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:50:55.024504 2106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-2/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-2/wal
uuid: "8db490d070364857829b9c3b8d5c4bc3"
format_stamp: "Formatted at 2026-08-10 18:50:55 on dist-test-slave-cbsf"
server_key: "a4ae893c779124d929861583727995d9"
server_key_iv: "7ac53e589eedef6f83dd117ec797b15c"
server_key_version: "encryptionkey@0"
I20260810 18:50:55.024585 2106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:50:55.050081 2106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:50:55.053434 2106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:50:55.053602 2106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:50:55.060998 2106 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.188:44755
I20260810 18:50:55.061308 2182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.188:44755 every 8 connection(s)
I20260810 18:50:55.062106 2106 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-2/data/info.pb
I20260810 18:50:55.065008 2183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:50:55.066942 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 2106
I20260810 18:50:55.067058 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387687857264-19106-0/minicluster-data/master-2/wal/instance
I20260810 18:50:55.067378 19106 external_mini_cluster.cc:1428] Setting key 8e84a3165dbb0ef303ac3fa95853bff3
I20260810 18:50:55.069406 2183 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.168.190" port: 34471 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:50:55 dist-test-slave-cbsf krb5kdc[1908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387855, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.188@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
Aug 10 18:50:55 dist-test-slave-cbsf krb5kdc[1908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.168.188@KRBTEST.COM
I20260810 18:50:55.081571 2004 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.168.188" port: 44755 } attempt: 3
I20260810 18:50:55.082091 2008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:55.071638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41448 (local address 127.18.168.190:34471)
0810 18:50:55.071783 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:55.071787 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:55.072156 (+ 369us) server_negotiation.cc:407] Connection header received
0810 18:50:55.072274 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:55.072277 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:55.072323 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:55.072397 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:50:55.073270 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:55.073753 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:55.074836 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:55.075141 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:55.075255 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:55.077768 (+ 2513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:50:55.077772 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:50:55.077775 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:50:55.077806 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:50:55.079544 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:55.080280 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:55.080283 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:55.080285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:55.080374 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:55.080823 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:55.080830 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:55.080832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:55.081100 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:50:55.081212 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:55.081747 (+ 535us) server_negotiation.cc:299] Negotiation successful
0810 18:50:55.081878 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 18:50:55.083081 2096 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.168.188" port: 44755 } attempt: 3
I20260810 18:50:55.083576 2187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:55.071536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:34471 (local address 127.0.0.1:41448)
0810 18:50:55.071999 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:55.072058 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 18:50:55.072176 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:50:55.072473 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:50:55.072478 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:50:55.072926 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:50:55.073132 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:55.073143 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:55.073879 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:55.073882 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:55.074681 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:55.074691 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:55.075314 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:55.075320 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:55.075454 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:55.075908 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:50:55.075932 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:50:55.077592 (+ 1660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:50:55.079681 (+ 2089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:55.079689 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:55.079698 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:55.080135 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:55.080480 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:55.080486 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:55.080490 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:55.080677 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:55.081214 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:50:55.081219 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:50:55.081452 (+ 233us) client_negotiation.cc:769] Sending connection context
0810 18:50:55.082220 (+ 768us) client_negotiation.cc:240] Negotiation successful
0810 18:50:55.082563 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"mutex_wait_us":3,"spinlock_wait_cycles":5888,"thread_start_us":160,"threads_started":1}
I20260810 18:50:55.083577 2186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:55.068859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49186 (local address 127.18.168.188:44755)
0810 18:50:55.069301 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:55.069317 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:55.069352 (+ 35us) server_negotiation.cc:407] Connection header received
0810 18:50:55.070142 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:55.070162 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:55.070561 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:55.070890 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:50:55.071793 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:55.073261 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:55.074213 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:55.074664 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:55.074828 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:55.078282 (+ 3454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:50:55.078287 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:50:55.078297 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:50:55.078318 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:50:55.080135 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:55.080606 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:55.080611 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:55.080612 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:55.080735 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:55.081034 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:55.081038 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:55.081040 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:55.081443 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:50:55.081643 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:55.081919 (+ 276us) server_negotiation.cc:299] Negotiation successful
0810 18:50:55.082630 (+ 711us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":102,"server-negotiator.queue_time_us":300,"thread_start_us":142,"threads_started":1}
I20260810 18:50:55.084828 2183 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33095 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:50:55 dist-test-slave-cbsf krb5kdc[1908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387854, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for kudu/127.18.168.188@KRBTEST.COM
Aug 10 18:50:55 dist-test-slave-cbsf krb5kdc[1908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387854, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.189@KRBTEST.COM for kudu/127.18.168.188@KRBTEST.COM
I20260810 18:50:55.092288 2009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:55.081986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:44755 (local address 127.0.0.1:49196)
0810 18:50:55.082089 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:55.082104 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:50:55.082196 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:50:55.082807 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:50:55.082810 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:50:55.082970 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:50:55.083082 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:55.083089 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:55.084420 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:55.084424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:55.085155 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:55.085165 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:55.086577 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:55.086583 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:55.086675 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:55.087106 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:50:55.087116 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:50:55.088635 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:50:55.090629 (+ 1994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:55.090632 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:55.090634 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:55.090893 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:55.091218 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:55.091221 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:55.091222 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:55.091310 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:55.091773 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:50:55.091780 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:50:55.091883 (+ 103us) client_negotiation.cc:769] Sending connection context
0810 18:50:55.091974 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 18:50:55.092106 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
Aug 10 18:50:55 dist-test-slave-cbsf krb5kdc[1908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387855, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.188@KRBTEST.COM for kudu/127.18.168.189@KRBTEST.COM
Aug 10 18:50:55 dist-test-slave-cbsf krb5kdc[1908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:50:55.093202 2188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:55.082251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49196 (local address 127.18.168.188:44755)
0810 18:50:55.082611 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:55.082618 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:55.082629 (+ 11us) server_negotiation.cc:407] Connection header received
0810 18:50:55.082657 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:55.082659 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:55.082690 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:55.082790 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:50:55.083251 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:55.084289 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:55.086115 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:55.086446 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:55.086877 (+ 431us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:55.088841 (+ 1964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:50:55.088844 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:50:55.088847 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:50:55.088870 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:50:55.090524 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:55.091006 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:55.091009 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:55.091010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:55.091100 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:55.091406 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:55.091410 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:55.091412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:55.091634 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:50:55.092575 (+ 941us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:55.092844 (+ 269us) server_negotiation.cc:299] Negotiation successful
0810 18:50:55.093006 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":129,"threads_started":1}
I20260810 18:50:55.094344 2100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:55.083549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:44755 (local address 127.0.0.1:49198)
0810 18:50:55.083720 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:55.083732 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 18:50:55.083811 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:50:55.084966 (+ 1155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:50:55.084969 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:50:55.085204 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:50:55.085318 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:55.085326 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:55.086227 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:55.086230 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:55.087026 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:55.087040 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:55.087985 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:55.087988 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:55.088087 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:55.088501 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:50:55.088523 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:50:55.089960 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:50:55.092276 (+ 2316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:55.092279 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:55.092280 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:55.092561 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:55.093021 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:55.093024 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:55.093026 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:55.093121 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:55.093745 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:50:55.093750 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:50:55.093862 (+ 112us) client_negotiation.cc:769] Sending connection context
0810 18:50:55.093968 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 18:50:55.094115 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"mutex_wait_us":3}
I20260810 18:50:55.096007 2186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:55.084190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49198 (local address 127.18.168.188:44755)
0810 18:50:55.084715 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:55.084719 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:55.084731 (+ 12us) server_negotiation.cc:407] Connection header received
0810 18:50:55.084761 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:55.084763 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:55.084799 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:55.084925 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:50:55.085412 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:55.086089 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:55.087438 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:55.087855 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:55.088300 (+ 445us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:55.090092 (+ 1792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:50:55.090097 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:50:55.090101 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:50:55.090129 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:50:55.092139 (+ 2010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:55.092817 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:55.092820 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:55.092821 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:55.092902 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:55.093402 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:55.093405 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:55.093407 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:55.093609 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:50:55.093742 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:55.094933 (+ 1191us) server_negotiation.cc:299] Negotiation successful
0810 18:50:55.095838 (+ 905us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":76,"threads_started":1}
I20260810 18:50:55.097148 2004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f8de7a0e6964ff586e98164489935d4: Bootstrap starting.
I20260810 18:50:55.097817 2187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:55.085264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.189:33095 (local address 127.0.0.1:40370)
0810 18:50:55.085483 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:55.085496 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:50:55.085565 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:50:55.085787 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:50:55.085790 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:50:55.085942 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:50:55.086043 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:55.086048 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:55.087332 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:55.087337 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:55.088883 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:55.088894 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:55.090175 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:55.090178 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:55.090286 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:55.090943 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:50:55.090963 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:50:55.093053 (+ 2090us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:50:55.095711 (+ 2658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:55.095715 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:55.095717 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:55.095965 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:55.096489 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:55.096492 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:55.096494 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:55.096634 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:55.097360 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:50:55.097363 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:50:55.097464 (+ 101us) client_negotiation.cc:769] Sending connection context
0810 18:50:55.097565 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 18:50:55.097684 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155}
I20260810 18:50:55.098173 2008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:55.086884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41462 (local address 127.18.168.190:34471)
0810 18:50:55.086980 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:55.086983 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:55.087286 (+ 303us) server_negotiation.cc:407] Connection header received
0810 18:50:55.087378 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:55.087380 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:55.087419 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:55.087491 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:50:55.088319 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:55.089038 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:55.090339 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:55.090676 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:55.090750 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:55.094008 (+ 3258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:50:55.094013 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:50:55.094015 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:50:55.094040 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:50:55.095797 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:55.096573 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:55.096580 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:55.096583 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:55.096662 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:55.097042 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:55.097045 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:55.097047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:55.097263 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:50:55.097451 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:55.097802 (+ 351us) server_negotiation.cc:299] Negotiation successful
0810 18:50:55.098040 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 18:50:55.098182 2104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:55.085375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40370 (local address 127.18.168.189:33095)
0810 18:50:55.085475 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:55.085479 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:55.085537 (+ 58us) server_negotiation.cc:407] Connection header received
0810 18:50:55.085634 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:55.085636 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:55.085681 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:55.085757 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:50:55.086189 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:55.087214 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:55.089051 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:55.089945 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:55.090041 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:55.093213 (+ 3172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:50:55.093220 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:50:55.093225 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:50:55.093255 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:50:55.095607 (+ 2352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:55.096098 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:55.096101 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:55.096103 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:55.096187 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:55.096811 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:55.096815 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:55.096816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:55.097208 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:50:55.097311 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:55.097664 (+ 353us) server_negotiation.cc:299] Negotiation successful
0810 18:50:55.097804 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":42}
I20260810 18:50:55.099260 2183 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.168.188" port: 44755 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:50:55.100092 2004 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f8de7a0e6964ff586e98164489935d4: Neither blocks nor log segments found. Creating new log.
I20260810 18:50:55.101094 2004 log.cc:824] T 00000000000000000000000000000000 P 1f8de7a0e6964ff586e98164489935d4: Log is configured to *not* fsync() on all Append() calls
I20260810 18:50:55.103267 2004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f8de7a0e6964ff586e98164489935d4: No bootstrap required, opened a new log
I20260810 18:50:55.104216 2096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ff6e91786b24e6a96fafa79313db149: Bootstrap starting.
Aug 10 18:50:55 dist-test-slave-cbsf krb5kdc[1908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387855, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.188@KRBTEST.COM for kudu/127.18.168.188@KRBTEST.COM
Aug 10 18:50:55 dist-test-slave-cbsf krb5kdc[1908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.168.189@KRBTEST.COM
I20260810 18:50:55.106936 2096 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ff6e91786b24e6a96fafa79313db149: Neither blocks nor log segments found. Creating new log.
I20260810 18:50:55.107801 2096 log.cc:824] T 00000000000000000000000000000000 P 3ff6e91786b24e6a96fafa79313db149: Log is configured to *not* fsync() on all Append() calls
I20260810 18:50:55.107874 2004 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f8de7a0e6964ff586e98164489935d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f8de7a0e6964ff586e98164489935d4" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 34471 } } peers { permanent_uuid: "3ff6e91786b24e6a96fafa79313db149" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33095 } } peers { permanent_uuid: "8db490d070364857829b9c3b8d5c4bc3" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 44755 } }
I20260810 18:50:55.108264 2004 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f8de7a0e6964ff586e98164489935d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:50:55.108384 2004 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f8de7a0e6964ff586e98164489935d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f8de7a0e6964ff586e98164489935d4, State: Initialized, Role: FOLLOWER
I20260810 18:50:55.109084 2004 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f8de7a0e6964ff586e98164489935d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f8de7a0e6964ff586e98164489935d4" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 34471 } } peers { permanent_uuid: "3ff6e91786b24e6a96fafa79313db149" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33095 } } peers { permanent_uuid: "8db490d070364857829b9c3b8d5c4bc3" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 44755 } }
I20260810 18:50:55.109472 2187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:55.099742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:44755 (local address 127.0.0.1:49208)
0810 18:50:55.100290 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:55.100304 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:50:55.100386 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:50:55.100644 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:50:55.100646 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:50:55.100832 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:50:55.100921 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:55.100930 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:55.101799 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:55.101802 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:55.102538 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:55.102547 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:55.103033 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:55.103037 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:55.103105 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:55.103617 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:50:55.103635 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:50:55.105744 (+ 2109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:50:55.107847 (+ 2103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:55.107853 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:55.107856 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:55.108082 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:55.108370 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:55.108375 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:55.108378 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:55.108451 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:55.109009 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:50:55.109012 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:50:55.109108 (+ 96us) client_negotiation.cc:769] Sending connection context
0810 18:50:55.109187 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 18:50:55.109304 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480}
I20260810 18:50:55.110011 2186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:55.099933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49208 (local address 127.18.168.188:44755)
0810 18:50:55.100015 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:55.100018 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:55.100459 (+ 441us) server_negotiation.cc:407] Connection header received
0810 18:50:55.100482 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:55.100484 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:55.100513 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:55.100566 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:50:55.101030 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:55.101700 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:55.102642 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:55.102931 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:55.103362 (+ 431us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:55.105878 (+ 2516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:50:55.105881 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:50:55.105883 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:50:55.105905 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:50:55.107733 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:55.108191 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:55.108195 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:55.108197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:55.108276 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:55.108592 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:55.108595 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:55.108596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:55.108899 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:50:55.109660 (+ 761us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:55.109729 (+ 69us) server_negotiation.cc:299] Negotiation successful
0810 18:50:55.109848 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 18:50:55.110157 2096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ff6e91786b24e6a96fafa79313db149: No bootstrap required, opened a new log
W20260810 18:50:55.110853 1950 tablet.cc:2369] T 00000000000000000000000000000000 P 1f8de7a0e6964ff586e98164489935d4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:50:55.111025 2004 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f8de7a0e6964ff586e98164489935d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:50:55.112879 2104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:55.099825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40384 (local address 127.18.168.189:33095)
0810 18:50:55.099928 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:55.099934 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:55.099949 (+ 15us) server_negotiation.cc:407] Connection header received
0810 18:50:55.099987 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:55.099992 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:55.100038 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:55.100128 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:50:55.100743 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:55.101462 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:55.102499 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:55.102798 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:55.102863 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:55.105892 (+ 3029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:50:55.105895 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:50:55.105897 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:50:55.105918 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:50:55.108086 (+ 2168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:55.108589 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:55.108592 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:55.108593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:55.108704 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:55.109109 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:55.109112 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:55.109113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:55.109311 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:50:55.109467 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:55.109986 (+ 519us) server_negotiation.cc:299] Negotiation successful
0810 18:50:55.110116 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 18:50:55.113400 2192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f8de7a0e6964ff586e98164489935d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f8de7a0e6964ff586e98164489935d4" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 34471 } } peers { permanent_uuid: "3ff6e91786b24e6a96fafa79313db149" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33095 } } peers { permanent_uuid: "8db490d070364857829b9c3b8d5c4bc3" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 44755 } } }
I20260810 18:50:55.113613 2192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f8de7a0e6964ff586e98164489935d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:50:55.114970 2183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8db490d070364857829b9c3b8d5c4bc3: Bootstrap starting.
I20260810 18:50:55.115435 2096 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ff6e91786b24e6a96fafa79313db149 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f8de7a0e6964ff586e98164489935d4" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 34471 } } peers { permanent_uuid: "3ff6e91786b24e6a96fafa79313db149" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33095 } } peers { permanent_uuid: "8db490d070364857829b9c3b8d5c4bc3" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 44755 } }
I20260810 18:50:55.115669 2096 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ff6e91786b24e6a96fafa79313db149 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:50:55.115741 2096 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ff6e91786b24e6a96fafa79313db149 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ff6e91786b24e6a96fafa79313db149, State: Initialized, Role: FOLLOWER
I20260810 18:50:55.116175 2096 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ff6e91786b24e6a96fafa79313db149 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f8de7a0e6964ff586e98164489935d4" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 34471 } } peers { permanent_uuid: "3ff6e91786b24e6a96fafa79313db149" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33095 } } peers { permanent_uuid: "8db490d070364857829b9c3b8d5c4bc3" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 44755 } }
I20260810 18:50:55.117197 2204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ff6e91786b24e6a96fafa79313db149 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f8de7a0e6964ff586e98164489935d4" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 34471 } } peers { permanent_uuid: "3ff6e91786b24e6a96fafa79313db149" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33095 } } peers { permanent_uuid: "8db490d070364857829b9c3b8d5c4bc3" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 44755 } } }
I20260810 18:50:55.117470 2204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ff6e91786b24e6a96fafa79313db149 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:50:55.117542 2183 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8db490d070364857829b9c3b8d5c4bc3: Neither blocks nor log segments found. Creating new log.
I20260810 18:50:55.117522 2096 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ff6e91786b24e6a96fafa79313db149 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:50:55.118296 2183 log.cc:824] T 00000000000000000000000000000000 P 8db490d070364857829b9c3b8d5c4bc3: Log is configured to *not* fsync() on all Append() calls
I20260810 18:50:55.120234 2183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8db490d070364857829b9c3b8d5c4bc3: No bootstrap required, opened a new log
I20260810 18:50:55.122031 19106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260810 18:50:55.122539 2217 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3ff6e91786b24e6a96fafa79313db149: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:50:55.122620 2217 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260810 18:50:55.123847 2203 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1f8de7a0e6964ff586e98164489935d4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:50:55.123987 2203 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:50:55.123921 2183 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8db490d070364857829b9c3b8d5c4bc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f8de7a0e6964ff586e98164489935d4" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 34471 } } peers { permanent_uuid: "3ff6e91786b24e6a96fafa79313db149" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33095 } } peers { permanent_uuid: "8db490d070364857829b9c3b8d5c4bc3" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 44755 } }
I20260810 18:50:55.124145 2183 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8db490d070364857829b9c3b8d5c4bc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:50:55.124197 2183 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8db490d070364857829b9c3b8d5c4bc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8db490d070364857829b9c3b8d5c4bc3, State: Initialized, Role: FOLLOWER
I20260810 18:50:55.124650 2183 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8db490d070364857829b9c3b8d5c4bc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f8de7a0e6964ff586e98164489935d4" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 34471 } } peers { permanent_uuid: "3ff6e91786b24e6a96fafa79313db149" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33095 } } peers { permanent_uuid: "8db490d070364857829b9c3b8d5c4bc3" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 44755 } }
I20260810 18:50:55.125350 2225 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8db490d070364857829b9c3b8d5c4bc3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f8de7a0e6964ff586e98164489935d4" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 34471 } } peers { permanent_uuid: "3ff6e91786b24e6a96fafa79313db149" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33095 } } peers { permanent_uuid: "8db490d070364857829b9c3b8d5c4bc3" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 44755 } } }
I20260810 18:50:55.117426 1923 master_runner.cc:186] Error getting master registration for 127.18.168.189:33095: OK, instance_id { permanent_uuid: "3ff6e91786b24e6a96fafa79313db149" instance_seqno: 1786387854826076 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 18:50:55.125551 1923 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 18:50:55.125554 2225 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8db490d070364857829b9c3b8d5c4bc3 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:50:55.126546 2183 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8db490d070364857829b9c3b8d5c4bc3 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 18:50:55.131702 2239 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8db490d070364857829b9c3b8d5c4bc3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:50:55.131783 2239 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:50:55.128518 2012 master_runner.cc:186] Error getting master registration for 127.18.168.188:44755: OK, instance_id { permanent_uuid: "8db490d070364857829b9c3b8d5c4bc3" instance_seqno: 1786387855058956 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 18:50:55.132387 2012 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 18:50:55.135067 2106 master_runner.cc:263] No leader master found from master 8db490d070364857829b9c3b8d5c4bc3
I20260810 18:50:55.135130 2106 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 18:50:55.148675 2104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:55.129105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40388 (local address 127.18.168.189:33095)
0810 18:50:55.129406 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:55.129410 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:55.131958 (+ 2548us) server_negotiation.cc:407] Connection header received
0810 18:50:55.132053 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:55.132056 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:55.132118 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:55.132198 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:50:55.133633 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:55.134411 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:55.135660 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:55.136023 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:55.136135 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:55.144205 (+ 8070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:50:55.144209 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:50:55.144212 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:50:55.144241 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:50:55.146843 (+ 2602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:55.147322 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:55.147326 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:55.147328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:55.147414 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:55.147728 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:55.147731 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:55.147733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:55.147937 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:50:55.148033 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:55.148373 (+ 340us) server_negotiation.cc:299] Negotiation successful
0810 18:50:55.148502 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205}
I20260810 18:50:55.158969 2186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:55.132149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49220 (local address 127.18.168.188:44755)
0810 18:50:55.132354 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:55.132359 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:55.138231 (+ 5872us) server_negotiation.cc:407] Connection header received
0810 18:50:55.138744 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:55.138747 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:55.138790 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:55.138836 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:50:55.141245 (+ 2409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:55.142020 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:55.145663 (+ 3643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:55.145992 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:55.146069 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:55.152936 (+ 6867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:50:55.152940 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:50:55.152943 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:50:55.152969 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:50:55.155008 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:55.156558 (+ 1550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:55.156561 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:55.156563 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:55.156645 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:55.157368 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:55.157371 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:55.157373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:55.157601 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:50:55.157713 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:55.158689 (+ 976us) server_negotiation.cc:299] Negotiation successful
0810 18:50:55.158811 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260810 18:50:55.159557 2008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:55.130073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41476 (local address 127.18.168.190:34471)
0810 18:50:55.130438 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:55.130442 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:55.138022 (+ 7580us) server_negotiation.cc:407] Connection header received
0810 18:50:55.138492 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:55.138495 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:55.138561 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:55.138611 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:50:55.140297 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:55.141108 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:55.144633 (+ 3525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:55.145074 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:55.145169 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:55.153348 (+ 8179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:50:55.153354 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:50:55.153358 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:50:55.153389 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:50:55.155753 (+ 2364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:55.156809 (+ 1056us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:55.156813 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:55.156815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:55.156896 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:55.157788 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:55.157792 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:55.157793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:55.158003 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:50:55.158117 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:55.159272 (+ 1155us) server_negotiation.cc:299] Negotiation successful
0810 18:50:55.159399 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303}
I20260810 18:50:55.185876 2225 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8db490d070364857829b9c3b8d5c4bc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:50:55.186100 2225 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8db490d070364857829b9c3b8d5c4bc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f8de7a0e6964ff586e98164489935d4" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 34471 } } peers { permanent_uuid: "3ff6e91786b24e6a96fafa79313db149" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33095 } } peers { permanent_uuid: "8db490d070364857829b9c3b8d5c4bc3" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 44755 } }
I20260810 18:50:55.187240 2225 leader_election.cc:288] T 00000000000000000000000000000000 P 8db490d070364857829b9c3b8d5c4bc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f8de7a0e6964ff586e98164489935d4 (127.18.168.190:34471), 3ff6e91786b24e6a96fafa79313db149 (127.18.168.189:33095)
I20260810 18:50:55.187867 1972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8db490d070364857829b9c3b8d5c4bc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f8de7a0e6964ff586e98164489935d4" is_pre_election: true
I20260810 18:50:55.188201 1972 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1f8de7a0e6964ff586e98164489935d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8db490d070364857829b9c3b8d5c4bc3 in term 0.
I20260810 18:50:55.188668 2124 leader_election.cc:302] T 00000000000000000000000000000000 P 8db490d070364857829b9c3b8d5c4bc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f8de7a0e6964ff586e98164489935d4, 8db490d070364857829b9c3b8d5c4bc3; no voters:
I20260810 18:50:55.189375 2063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8db490d070364857829b9c3b8d5c4bc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ff6e91786b24e6a96fafa79313db149" is_pre_election: true
I20260810 18:50:55.189642 2063 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3ff6e91786b24e6a96fafa79313db149 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8db490d070364857829b9c3b8d5c4bc3 in term 0.
I20260810 18:50:55.190212 2225 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8db490d070364857829b9c3b8d5c4bc3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:50:55.190320 2225 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8db490d070364857829b9c3b8d5c4bc3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:50:55.190371 2225 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8db490d070364857829b9c3b8d5c4bc3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:50:55.191712 2225 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8db490d070364857829b9c3b8d5c4bc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f8de7a0e6964ff586e98164489935d4" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 34471 } } peers { permanent_uuid: "3ff6e91786b24e6a96fafa79313db149" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33095 } } peers { permanent_uuid: "8db490d070364857829b9c3b8d5c4bc3" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 44755 } }
I20260810 18:50:55.192024 2225 leader_election.cc:288] T 00000000000000000000000000000000 P 8db490d070364857829b9c3b8d5c4bc3 [CANDIDATE]: Term 1 election: Requested vote from peers 1f8de7a0e6964ff586e98164489935d4 (127.18.168.190:34471), 3ff6e91786b24e6a96fafa79313db149 (127.18.168.189:33095)
I20260810 18:50:55.192430 1972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8db490d070364857829b9c3b8d5c4bc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f8de7a0e6964ff586e98164489935d4"
I20260810 18:50:55.192545 1972 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f8de7a0e6964ff586e98164489935d4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:50:55.193849 1972 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1f8de7a0e6964ff586e98164489935d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8db490d070364857829b9c3b8d5c4bc3 in term 1.
I20260810 18:50:55.194164 2124 leader_election.cc:302] T 00000000000000000000000000000000 P 8db490d070364857829b9c3b8d5c4bc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f8de7a0e6964ff586e98164489935d4, 8db490d070364857829b9c3b8d5c4bc3; no voters:
I20260810 18:50:55.194581 2063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8db490d070364857829b9c3b8d5c4bc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ff6e91786b24e6a96fafa79313db149"
I20260810 18:50:55.194725 2063 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ff6e91786b24e6a96fafa79313db149 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:50:55.195986 2063 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3ff6e91786b24e6a96fafa79313db149 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8db490d070364857829b9c3b8d5c4bc3 in term 1.
I20260810 18:50:55.198747 2225 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8db490d070364857829b9c3b8d5c4bc3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:50:55.198987 2225 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8db490d070364857829b9c3b8d5c4bc3 [term 1 LEADER]: Becoming Leader. State: Replica: 8db490d070364857829b9c3b8d5c4bc3, State: Running, Role: LEADER
I20260810 18:50:55.199338 2225 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8db490d070364857829b9c3b8d5c4bc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f8de7a0e6964ff586e98164489935d4" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 34471 } } peers { permanent_uuid: "3ff6e91786b24e6a96fafa79313db149" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33095 } } peers { permanent_uuid: "8db490d070364857829b9c3b8d5c4bc3" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 44755 } }
I20260810 18:50:55.201124 2225 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8db490d070364857829b9c3b8d5c4bc3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8db490d070364857829b9c3b8d5c4bc3. Latest consensus state: current_term: 1 leader_uuid: "8db490d070364857829b9c3b8d5c4bc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f8de7a0e6964ff586e98164489935d4" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 34471 } } peers { permanent_uuid: "3ff6e91786b24e6a96fafa79313db149" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33095 } } peers { permanent_uuid: "8db490d070364857829b9c3b8d5c4bc3" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 44755 } } }
I20260810 18:50:55.201232 2225 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8db490d070364857829b9c3b8d5c4bc3 [sys.catalog]: This master's current role is: LEADER
I20260810 18:50:55.201594 2249 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:50:55.202534 2249 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:50:55.211601 2063 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3ff6e91786b24e6a96fafa79313db149 [term 1 FOLLOWER]: Refusing update from remote peer 8db490d070364857829b9c3b8d5c4bc3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:50:55.212354 2225 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8db490d070364857829b9c3b8d5c4bc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ff6e91786b24e6a96fafa79313db149" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:50:55.216869 2204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ff6e91786b24e6a96fafa79313db149 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8db490d070364857829b9c3b8d5c4bc3. Latest consensus state: current_term: 1 leader_uuid: "8db490d070364857829b9c3b8d5c4bc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f8de7a0e6964ff586e98164489935d4" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 34471 } } peers { permanent_uuid: "3ff6e91786b24e6a96fafa79313db149" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33095 } } peers { permanent_uuid: "8db490d070364857829b9c3b8d5c4bc3" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 44755 } } }
I20260810 18:50:55.216989 2204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ff6e91786b24e6a96fafa79313db149 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:50:55.217973 1972 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1f8de7a0e6964ff586e98164489935d4 [term 1 FOLLOWER]: Refusing update from remote peer 8db490d070364857829b9c3b8d5c4bc3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:50:55.218676 2225 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8db490d070364857829b9c3b8d5c4bc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f8de7a0e6964ff586e98164489935d4" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 34471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:50:55.224988 2192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f8de7a0e6964ff586e98164489935d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8db490d070364857829b9c3b8d5c4bc3. Latest consensus state: current_term: 1 leader_uuid: "8db490d070364857829b9c3b8d5c4bc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f8de7a0e6964ff586e98164489935d4" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 34471 } } peers { permanent_uuid: "3ff6e91786b24e6a96fafa79313db149" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33095 } } peers { permanent_uuid: "8db490d070364857829b9c3b8d5c4bc3" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 44755 } } }
I20260810 18:50:55.225116 2192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f8de7a0e6964ff586e98164489935d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:50:55.235412 2192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f8de7a0e6964ff586e98164489935d4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8db490d070364857829b9c3b8d5c4bc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f8de7a0e6964ff586e98164489935d4" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 34471 } } peers { permanent_uuid: "3ff6e91786b24e6a96fafa79313db149" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33095 } } peers { permanent_uuid: "8db490d070364857829b9c3b8d5c4bc3" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 44755 } } }
I20260810 18:50:55.235527 2192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f8de7a0e6964ff586e98164489935d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:50:55.236559 2246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8db490d070364857829b9c3b8d5c4bc3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8db490d070364857829b9c3b8d5c4bc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f8de7a0e6964ff586e98164489935d4" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 34471 } } peers { permanent_uuid: "3ff6e91786b24e6a96fafa79313db149" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33095 } } peers { permanent_uuid: "8db490d070364857829b9c3b8d5c4bc3" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 44755 } } }
I20260810 18:50:55.236709 2246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8db490d070364857829b9c3b8d5c4bc3 [sys.catalog]: This master's current role is: LEADER
I20260810 18:50:55.237816 2250 mvcc.cc:205] Tried to move back new op lower bound from 7317044654930481152 to 7317044654899654656. Current Snapshot: MvccSnapshot[applied={T|T < 7317044654930481152}]
I20260810 18:50:55.238185 2254 mvcc.cc:205] Tried to move back new op lower bound from 7317044654930481152 to 7317044654899654656. Current Snapshot: MvccSnapshot[applied={T|T < 7317044654930481152}]
I20260810 18:50:55.238840 2225 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8db490d070364857829b9c3b8d5c4bc3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8db490d070364857829b9c3b8d5c4bc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f8de7a0e6964ff586e98164489935d4" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 34471 } } peers { permanent_uuid: "3ff6e91786b24e6a96fafa79313db149" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33095 } } peers { permanent_uuid: "8db490d070364857829b9c3b8d5c4bc3" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 44755 } } }
I20260810 18:50:55.238953 2225 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8db490d070364857829b9c3b8d5c4bc3 [sys.catalog]: This master's current role is: LEADER
I20260810 18:50:55.239742 2204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ff6e91786b24e6a96fafa79313db149 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8db490d070364857829b9c3b8d5c4bc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f8de7a0e6964ff586e98164489935d4" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 34471 } } peers { permanent_uuid: "3ff6e91786b24e6a96fafa79313db149" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33095 } } peers { permanent_uuid: "8db490d070364857829b9c3b8d5c4bc3" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 44755 } } }
I20260810 18:50:55.239853 2204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ff6e91786b24e6a96fafa79313db149 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:50:55.241102 2251 mvcc.cc:205] Tried to move back new op lower bound from 7317044654930481152 to 7317044654899654656. Current Snapshot: MvccSnapshot[applied={T|T < 7317044654930481152}]
I20260810 18:50:55.244410 2249 catalog_manager.cc:1347] Generated new cluster ID: 9403bbf7df6942c39630795da36f17e8
I20260810 18:50:55.244475 2249 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:50:55.272238 2249 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:50:55.273522 2249 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:50:55.292385 2249 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8db490d070364857829b9c3b8d5c4bc3: Generated new TSK 0
I20260810 18:50:55.296002 2249 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:50:56.124352 2217 catalog_manager.cc:1259] Loaded cluster ID: 9403bbf7df6942c39630795da36f17e8
I20260810 18:50:56.124516 2217 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3ff6e91786b24e6a96fafa79313db149: loading cluster ID for follower catalog manager: success
I20260810 18:50:56.125416 2203 catalog_manager.cc:1259] Loaded cluster ID: 9403bbf7df6942c39630795da36f17e8
I20260810 18:50:56.125497 2203 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1f8de7a0e6964ff586e98164489935d4: loading cluster ID for follower catalog manager: success
I20260810 18:50:56.127046 2217 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3ff6e91786b24e6a96fafa79313db149: acquiring CA information for follower catalog manager: success
I20260810 18:50:56.127619 2203 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1f8de7a0e6964ff586e98164489935d4: acquiring CA information for follower catalog manager: success
I20260810 18:50:56.128733 2217 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3ff6e91786b24e6a96fafa79313db149: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 18:50:56.128892 2203 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1f8de7a0e6964ff586e98164489935d4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 18:50:56.371862 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 1923
I20260810 18:50:56.382396 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 2012
I20260810 18:50:56.390846 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 2106
2026-08-10T18:50:56Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3290 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:50:56 dist-test-slave-cbsf krb5kdc[2267](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:50:56 dist-test-slave-cbsf krb5kdc[2267](info): set up 2 sockets
Aug 10 18:50:56 dist-test-slave-cbsf krb5kdc[2267](info): commencing operation
krb5kdc: starting...
W20260810 18:50:57.503376 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:50:57 dist-test-slave-cbsf krb5kdc[2267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:50:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:50:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.168.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
I20260810 18:50:57.628350 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:33353
--webserver_interface=127.18.168.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:36029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:33353
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:50:57.738750 2281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:50:57.738978 2281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:50:57.739017 2281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:50:57.742990 2281 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:50:57.743040 2281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:50:57.743059 2281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:50:57.743075 2281 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:50:57.743090 2281 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:50:57.748144 2281 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:36029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:33353
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:33353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:50:57.749176 2281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:50:57.750169 2281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:50:57.758939 2290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:50:57.759013 2288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:50:57.758970 2287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:50:57.759213 2281 server_base.cc:1034] running on GCE node
I20260810 18:50:57.759845 2281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:50:57.760999 2281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:50:57.762171 2281 hybrid_clock.cc:648] HybridClock initialized: now 1786387857762150 us; error 29 us; skew 500 ppm
Aug 10 18:50:57 dist-test-slave-cbsf krb5kdc[2267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387857, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:50:57.764984 2281 init.cc:377] Logged in from keytab as kudu/127.18.168.190@KRBTEST.COM (short username kudu)
I20260810 18:50:57.765842 2281 webserver.cc:460] Webserver started at http://127.18.168.190:39363/ using document root <none> and password file <none>
I20260810 18:50:57.766412 2281 fs_manager.cc:359] Metadata directory not provided
I20260810 18:50:57.766506 2281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:50:57.766752 2281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:50:57.768920 2281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "fb41768327a94bb5a84364fd185af24c"
format_stamp: "Formatted at 2026-08-10 18:50:57 on dist-test-slave-cbsf"
server_key: "545a21cfb8cae05b6a27cc26005d8f9d"
server_key_iv: "09dd2ea6a6c6570d873c9d4e383bb9e4"
server_key_version: "encryptionkey@0"
I20260810 18:50:57.769510 2281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "fb41768327a94bb5a84364fd185af24c"
format_stamp: "Formatted at 2026-08-10 18:50:57 on dist-test-slave-cbsf"
server_key: "545a21cfb8cae05b6a27cc26005d8f9d"
server_key_iv: "09dd2ea6a6c6570d873c9d4e383bb9e4"
server_key_version: "encryptionkey@0"
I20260810 18:50:57.773296 2281 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:50:57.775746 2298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:57.777025 2281 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:50:57.777186 2281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "fb41768327a94bb5a84364fd185af24c"
format_stamp: "Formatted at 2026-08-10 18:50:57 on dist-test-slave-cbsf"
server_key: "545a21cfb8cae05b6a27cc26005d8f9d"
server_key_iv: "09dd2ea6a6c6570d873c9d4e383bb9e4"
server_key_version: "encryptionkey@0"
I20260810 18:50:57.777302 2281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:50:57.803429 2281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:50:57.806361 2281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:50:57.806572 2281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:50:57.813557 2281 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:33353
I20260810 18:50:57.813577 2356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:33353 every 8 connection(s)
I20260810 18:50:57.814497 2281 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:50:57.814935 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 2281
I20260810 18:50:57.815088 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:50:57.815387 19106 external_mini_cluster.cc:1428] Setting key 7e700be592e0ca71400de60c2a77a5b7
I20260810 18:50:57.818076 2357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 18:50:57 dist-test-slave-cbsf krb5kdc[2267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:50:57.824882 2357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb41768327a94bb5a84364fd185af24c: Bootstrap starting.
I20260810 18:50:57.827389 2357 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb41768327a94bb5a84364fd185af24c: Neither blocks nor log segments found. Creating new log.
I20260810 18:50:57.828294 2357 log.cc:824] T 00000000000000000000000000000000 P fb41768327a94bb5a84364fd185af24c: Log is configured to *not* fsync() on all Append() calls
I20260810 18:50:57.830261 2361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:57.816624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52028 (local address 127.18.168.190:33353)
0810 18:50:57.817111 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:57.817130 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:57.817185 (+ 55us) server_negotiation.cc:407] Connection header received
0810 18:50:57.818025 (+ 840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:57.818048 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:57.818455 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:57.818797 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:50:57.819600 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:57.821124 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:57.821934 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:57.822589 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:57.822740 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:57.825480 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:50:57.825485 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:50:57.825497 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:50:57.825522 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:50:57.827150 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:57.827675 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:57.827680 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:57.827685 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:57.827820 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:57.828116 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:57.828118 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:57.828120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:57.828555 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:50:57.828775 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:57.829070 (+ 295us) server_negotiation.cc:299] Negotiation successful
0810 18:50:57.829374 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":110,"threads_started":1}
I20260810 18:50:57.830586 2357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb41768327a94bb5a84364fd185af24c: No bootstrap required, opened a new log
I20260810 18:50:57.833243 19106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:50:57.834862 2357 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb41768327a94bb5a84364fd185af24c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb41768327a94bb5a84364fd185af24c" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 33353 } }
I20260810 18:50:57.835069 2357 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb41768327a94bb5a84364fd185af24c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:50:57.835156 2357 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb41768327a94bb5a84364fd185af24c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb41768327a94bb5a84364fd185af24c, State: Initialized, Role: FOLLOWER
I20260810 18:50:57.835605 2357 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb41768327a94bb5a84364fd185af24c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb41768327a94bb5a84364fd185af24c" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 33353 } }
I20260810 18:50:57.835763 2357 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb41768327a94bb5a84364fd185af24c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:50:57.835850 2357 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb41768327a94bb5a84364fd185af24c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:50:57.835968 2357 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb41768327a94bb5a84364fd185af24c [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:50:57.836906 2357 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb41768327a94bb5a84364fd185af24c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb41768327a94bb5a84364fd185af24c" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 33353 } }
I20260810 18:50:57.837255 2357 leader_election.cc:302] T 00000000000000000000000000000000 P fb41768327a94bb5a84364fd185af24c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb41768327a94bb5a84364fd185af24c; no voters:
I20260810 18:50:57.837563 2357 leader_election.cc:288] T 00000000000000000000000000000000 P fb41768327a94bb5a84364fd185af24c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:50:57.837700 2363 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb41768327a94bb5a84364fd185af24c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:50:57.837987 2363 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb41768327a94bb5a84364fd185af24c [term 1 LEADER]: Becoming Leader. State: Replica: fb41768327a94bb5a84364fd185af24c, State: Running, Role: LEADER
I20260810 18:50:57.838342 2363 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb41768327a94bb5a84364fd185af24c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb41768327a94bb5a84364fd185af24c" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 33353 } }
I20260810 18:50:57.838776 2357 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb41768327a94bb5a84364fd185af24c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:50:57.840060 2365 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb41768327a94bb5a84364fd185af24c [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb41768327a94bb5a84364fd185af24c. Latest consensus state: current_term: 1 leader_uuid: "fb41768327a94bb5a84364fd185af24c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb41768327a94bb5a84364fd185af24c" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 33353 } } }
I20260810 18:50:57.840196 2365 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb41768327a94bb5a84364fd185af24c [sys.catalog]: This master's current role is: LEADER
I20260810 18:50:57.840504 2364 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb41768327a94bb5a84364fd185af24c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb41768327a94bb5a84364fd185af24c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb41768327a94bb5a84364fd185af24c" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 33353 } } }
I20260810 18:50:57.840584 2372 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:50:57.840603 2364 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb41768327a94bb5a84364fd185af24c [sys.catalog]: This master's current role is: LEADER
I20260810 18:50:57.843631 2372 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:50:57.849272 2372 catalog_manager.cc:1347] Generated new cluster ID: 72a4789a7d804e82a7015cc25ebf6220
I20260810 18:50:57.849334 2372 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:50:57.859150 2372 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:50:57.860308 2372 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:50:57.866441 2372 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb41768327a94bb5a84364fd185af24c: Generated new TSK 0
I20260810 18:50:57.867161 2372 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:50:57.888835 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 2281
2026-08-10T18:50:57Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1498 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:50:57 dist-test-slave-cbsf krb5kdc[2388](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:50:57 dist-test-slave-cbsf krb5kdc[2388](info): set up 2 sockets
Aug 10 18:50:57 dist-test-slave-cbsf krb5kdc[2388](info): commencing operation
krb5kdc: starting...
W20260810 18:50:59.026090 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:50:59 dist-test-slave-cbsf krb5kdc[2388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:50:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:50:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.168.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
I20260810 18:50:59.152045 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:40999
--webserver_interface=127.18.168.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:39893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:40999
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:50:59.268215 2404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:50:59.268486 2404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:50:59.268559 2404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:50:59.272401 2404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:50:59.272480 2404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:50:59.272517 2404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:50:59.272560 2404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:50:59.272593 2404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:50:59.277774 2404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:39893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:40999
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:40999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:50:59.278769 2404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:50:59.279754 2404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:50:59.288791 2415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:50:59.288831 2412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:50:59.288841 2413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:50:59.288918 2404 server_base.cc:1034] running on GCE node
I20260810 18:50:59.289499 2404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:50:59.290551 2404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:50:59.291714 2404 hybrid_clock.cc:648] HybridClock initialized: now 1786387859291695 us; error 30 us; skew 500 ppm
Aug 10 18:50:59 dist-test-slave-cbsf krb5kdc[2388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387859, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:50:59.298216 2404 init.cc:377] Logged in from keytab as kudu/127.18.168.190@KRBTEST.COM (short username kudu)
I20260810 18:50:59.299193 2404 webserver.cc:460] Webserver started at http://127.18.168.190:40807/ using document root <none> and password file <none>
I20260810 18:50:59.299748 2404 fs_manager.cc:359] Metadata directory not provided
I20260810 18:50:59.299815 2404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:50:59.300009 2404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:50:59.302165 2404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "69089a0bf06544d4bbca5fc069539ac2"
format_stamp: "Formatted at 2026-08-10 18:50:59 on dist-test-slave-cbsf"
server_key: "9d15b64b5cca04f3ec706f523022d8e1"
server_key_iv: "da562ea5a3130604866f91c09367e839"
server_key_version: "encryptionkey@0"
I20260810 18:50:59.302709 2404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "69089a0bf06544d4bbca5fc069539ac2"
format_stamp: "Formatted at 2026-08-10 18:50:59 on dist-test-slave-cbsf"
server_key: "9d15b64b5cca04f3ec706f523022d8e1"
server_key_iv: "da562ea5a3130604866f91c09367e839"
server_key_version: "encryptionkey@0"
I20260810 18:50:59.306536 2404 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260810 18:50:59.308934 2422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:59.310164 2404 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:50:59.310325 2404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "69089a0bf06544d4bbca5fc069539ac2"
format_stamp: "Formatted at 2026-08-10 18:50:59 on dist-test-slave-cbsf"
server_key: "9d15b64b5cca04f3ec706f523022d8e1"
server_key_iv: "da562ea5a3130604866f91c09367e839"
server_key_version: "encryptionkey@0"
I20260810 18:50:59.310429 2404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:50:59.322310 2404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:50:59.325234 2404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:50:59.325459 2404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:50:59.332780 2404 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:40999
I20260810 18:50:59.333101 2481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:40999 every 8 connection(s)
I20260810 18:50:59.333758 2404 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:50:59.336458 2482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:50:59.338596 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 2404
I20260810 18:50:59.338701 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:50:59.338981 19106 external_mini_cluster.cc:1428] Setting key b73f9c6176e02ed9c65a45781a08f2cb
I20260810 18:50:59.342401 2482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2: Bootstrap starting.
I20260810 18:50:59.344900 2482 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2: Neither blocks nor log segments found. Creating new log.
I20260810 18:50:59.345732 2482 log.cc:824] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2: Log is configured to *not* fsync() on all Append() calls
Aug 10 18:50:59 dist-test-slave-cbsf krb5kdc[2388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:50:59.347843 2482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2: No bootstrap required, opened a new log
I20260810 18:50:59.352555 2482 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69089a0bf06544d4bbca5fc069539ac2" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 40999 } }
I20260810 18:50:59.352838 2482 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:50:59.352923 2482 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69089a0bf06544d4bbca5fc069539ac2, State: Initialized, Role: FOLLOWER
I20260810 18:50:59.353370 2482 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69089a0bf06544d4bbca5fc069539ac2" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 40999 } }
I20260810 18:50:59.353518 2485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:59.340258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39016 (local address 127.18.168.190:40999)
0810 18:50:59.340660 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:59.340676 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:59.340756 (+ 80us) server_negotiation.cc:407] Connection header received
0810 18:50:59.341482 (+ 726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:59.341500 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:59.341809 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:59.342056 (+ 247us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:50:59.342915 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:59.343922 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:59.345117 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:59.345595 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:59.345733 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:59.348404 (+ 2671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:50:59.348410 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:50:59.348425 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:50:59.348451 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:50:59.350433 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:59.350899 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:59.350905 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:59.350911 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:59.351051 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:59.351318 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:59.351321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:59.351323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:59.351826 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:50:59.352025 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:59.352216 (+ 191us) server_negotiation.cc:299] Negotiation successful
0810 18:50:59.352494 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":143,"threads_started":1}
I20260810 18:50:59.353545 2482 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:50:59.353606 2482 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:50:59.353703 2482 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:50:59.354543 2482 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69089a0bf06544d4bbca5fc069539ac2" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 40999 } }
I20260810 18:50:59.354900 2482 leader_election.cc:302] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69089a0bf06544d4bbca5fc069539ac2; no voters:
I20260810 18:50:59.355203 2482 leader_election.cc:288] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:50:59.355311 2488 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:50:59.355551 2488 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [term 1 LEADER]: Becoming Leader. State: Replica: 69089a0bf06544d4bbca5fc069539ac2, State: Running, Role: LEADER
I20260810 18:50:59.355865 2488 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69089a0bf06544d4bbca5fc069539ac2" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 40999 } }
I20260810 18:50:59.356506 2482 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:50:59.357043 2488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69089a0bf06544d4bbca5fc069539ac2. Latest consensus state: current_term: 1 leader_uuid: "69089a0bf06544d4bbca5fc069539ac2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69089a0bf06544d4bbca5fc069539ac2" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 40999 } } }
I20260810 18:50:59.357254 2488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [sys.catalog]: This master's current role is: LEADER
I20260810 18:50:59.357795 2489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69089a0bf06544d4bbca5fc069539ac2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69089a0bf06544d4bbca5fc069539ac2" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 40999 } } }
I20260810 18:50:59.357892 2489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [sys.catalog]: This master's current role is: LEADER
I20260810 18:50:59.358289 2498 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:50:59.361415 2498 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:50:59.366882 2498 catalog_manager.cc:1347] Generated new cluster ID: 9dc514d0f3ce4c8da0755be8b4a96313
I20260810 18:50:59.366943 2498 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:50:59.384256 2498 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:50:59.385212 2498 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:50:59.391644 2498 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2: Generated new TSK 0
I20260810 18:50:59.392307 2498 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.168.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab.
Entry for principal kudu/127.18.168.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab.
I20260810 18:50:59.428545 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.129:0
--local_ip_for_outbound_sockets=127.18.168.129
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:40999
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:39893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:50:59.541666 2513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:50:59.541941 2513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:50:59.542013 2513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:50:59.546713 2513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:50:59.546797 2513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:50:59.546885 2513 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.129
I20260810 18:50:59.551986 2513 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:39893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:40999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2513
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.129
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:50:59.553071 2513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:50:59.554080 2513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:50:59.563952 2519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:50:59.563967 2521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:50:59.563971 2518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:50:59.564016 2513 server_base.cc:1034] running on GCE node
I20260810 18:50:59.564617 2513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:50:59.565294 2513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:50:59.566455 2513 hybrid_clock.cc:648] HybridClock initialized: now 1786387859566426 us; error 41 us; skew 500 ppm
Aug 10 18:50:59 dist-test-slave-cbsf krb5kdc[2388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387859, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:50:59.569244 2513 init.cc:377] Logged in from keytab as kudu/127.18.168.129@KRBTEST.COM (short username kudu)
I20260810 18:50:59.569996 2513 webserver.cc:460] Webserver started at http://127.18.168.129:41297/ using document root <none> and password file <none>
I20260810 18:50:59.570505 2513 fs_manager.cc:359] Metadata directory not provided
I20260810 18:50:59.570559 2513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:50:59.570722 2513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:50:59.572609 2513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/data/instance:
uuid: "7c3c35e869e641a98116d03ab65f70ca"
format_stamp: "Formatted at 2026-08-10 18:50:59 on dist-test-slave-cbsf"
server_key: "2cf66e56f2f034576bd6dd15490fd60e"
server_key_iv: "c32eb7f3ba0919eb3ffb84505d19d9fe"
server_key_version: "encryptionkey@0"
I20260810 18:50:59.573230 2513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance:
uuid: "7c3c35e869e641a98116d03ab65f70ca"
format_stamp: "Formatted at 2026-08-10 18:50:59 on dist-test-slave-cbsf"
server_key: "2cf66e56f2f034576bd6dd15490fd60e"
server_key_iv: "c32eb7f3ba0919eb3ffb84505d19d9fe"
server_key_version: "encryptionkey@0"
I20260810 18:50:59.576836 2513 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:50:59.579174 2529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:59.580382 2513 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:50:59.580513 2513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/wal
uuid: "7c3c35e869e641a98116d03ab65f70ca"
format_stamp: "Formatted at 2026-08-10 18:50:59 on dist-test-slave-cbsf"
server_key: "2cf66e56f2f034576bd6dd15490fd60e"
server_key_iv: "c32eb7f3ba0919eb3ffb84505d19d9fe"
server_key_version: "encryptionkey@0"
I20260810 18:50:59.580613 2513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:50:59.609642 2513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:50:59.612677 2513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:50:59.612960 2513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:50:59.613603 2513 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:50:59.614501 2513 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:50:59.614549 2513 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:59.614625 2513 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:50:59.614666 2513 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:59.623940 2513 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.129:37577
I20260810 18:50:59.623980 2659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.129:37577 every 8 connection(s)
I20260810 18:50:59.624873 2513 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
I20260810 18:50:59.625265 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 2513
I20260810 18:50:59.625401 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance
I20260810 18:50:59.625728 19106 external_mini_cluster.cc:1428] Setting key 06dc447cd8da1e7d41fcf73f6325fc24
Aug 10 18:50:59 dist-test-slave-cbsf krb5kdc[2388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387859, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.168.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.130@KRBTEST.COM" created.
I20260810 18:50:59.640514 2485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:59.626751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:49953 (local address 127.18.168.190:40999)
0810 18:50:59.626903 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:59.626908 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:59.627851 (+ 943us) server_negotiation.cc:407] Connection header received
0810 18:50:59.629403 (+ 1552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:59.629406 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:59.629462 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:59.629544 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:50:59.630711 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:59.631272 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:59.633105 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:59.633555 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:59.633700 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:59.636765 (+ 3065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:50:59.636769 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:50:59.636772 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:50:59.636806 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:50:59.638416 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:59.638907 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:59.638910 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:59.638911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:59.638990 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:59.639303 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:59.639306 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:59.639307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:59.639614 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:50:59.639735 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:59.640207 (+ 472us) server_negotiation.cc:299] Negotiation successful
0810 18:50:59.640376 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 18:50:59.641687 2662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:59.627196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.129:49953)
0810 18:50:59.627706 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:59.627748 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 18:50:59.629185 (+ 1437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:50:59.629717 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:50:59.629724 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:50:59.630081 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:50:59.630548 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:59.630561 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:59.631415 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:59.631420 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:59.632961 (+ 1541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:59.632974 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:59.633783 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:59.633787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:59.633998 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:59.634625 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:50:59.634651 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:50:59.636526 (+ 1875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:50:59.638543 (+ 2017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:59.638547 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:59.638559 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:59.638810 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:59.639082 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:59.639084 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:59.639085 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:59.639215 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:59.639750 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:50:59.639754 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:50:59.639981 (+ 227us) client_negotiation.cc:769] Sending connection context
0810 18:50:59.640188 (+ 207us) client_negotiation.cc:240] Negotiation successful
0810 18:50:59.640554 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":146,"threads_started":1}
I20260810 18:50:59.643047 2660 heartbeater.cc:347] Connected to a master server at 127.18.168.190:40999
I20260810 18:50:59.643335 2660 heartbeater.cc:464] Registering TS with master...
I20260810 18:50:59.643909 2660 heartbeater.cc:511] Master 127.18.168.190:40999 requested a full tablet report, sending...
I20260810 18:50:59.645800 2440 ts_manager.cc:194] Registered new tserver with Master: 7c3c35e869e641a98116d03ab65f70ca (127.18.168.129:37577)
I20260810 18:50:59.647544 2440 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.129@KRBTEST.COM'} at 127.18.168.129:49953
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab.
Entry for principal kudu/127.18.168.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab.
I20260810 18:50:59.658262 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.130:0
--local_ip_for_outbound_sockets=127.18.168.130
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:40999
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:39893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:50:59.771942 2666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:50:59.772181 2666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:50:59.772243 2666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:50:59.776086 2666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:50:59.776139 2666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:50:59.776194 2666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.130
I20260810 18:50:59.781164 2666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:39893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:40999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.130
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:50:59.782094 2666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:50:59.783026 2666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:50:59.792783 2674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:50:59.792835 2671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:50:59.793038 2666 server_base.cc:1034] running on GCE node
W20260810 18:50:59.792783 2672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:50:59.793475 2666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:50:59.794164 2666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:50:59.795368 2666 hybrid_clock.cc:648] HybridClock initialized: now 1786387859795343 us; error 34 us; skew 500 ppm
Aug 10 18:50:59 dist-test-slave-cbsf krb5kdc[2388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387859, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:50:59.798250 2666 init.cc:377] Logged in from keytab as kudu/127.18.168.130@KRBTEST.COM (short username kudu)
I20260810 18:50:59.799093 2666 webserver.cc:460] Webserver started at http://127.18.168.130:39067/ using document root <none> and password file <none>
I20260810 18:50:59.799707 2666 fs_manager.cc:359] Metadata directory not provided
I20260810 18:50:59.799822 2666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:50:59.800058 2666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:50:59.802212 2666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/data/instance:
uuid: "436302659835484193079dea06d4e462"
format_stamp: "Formatted at 2026-08-10 18:50:59 on dist-test-slave-cbsf"
server_key: "102fe108fde9dcf0179f2fec6854b3a3"
server_key_iv: "9b39e195684ab7b07072f6f3d835150d"
server_key_version: "encryptionkey@0"
I20260810 18:50:59.802785 2666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance:
uuid: "436302659835484193079dea06d4e462"
format_stamp: "Formatted at 2026-08-10 18:50:59 on dist-test-slave-cbsf"
server_key: "102fe108fde9dcf0179f2fec6854b3a3"
server_key_iv: "9b39e195684ab7b07072f6f3d835150d"
server_key_version: "encryptionkey@0"
I20260810 18:50:59.806622 2666 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:50:59.809123 2682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:59.810345 2666 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:50:59.810490 2666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/wal
uuid: "436302659835484193079dea06d4e462"
format_stamp: "Formatted at 2026-08-10 18:50:59 on dist-test-slave-cbsf"
server_key: "102fe108fde9dcf0179f2fec6854b3a3"
server_key_iv: "9b39e195684ab7b07072f6f3d835150d"
server_key_version: "encryptionkey@0"
I20260810 18:50:59.810621 2666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:50:59.830577 2666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:50:59.834029 2666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:50:59.834249 2666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:50:59.834877 2666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:50:59.835700 2666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:50:59.835744 2666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:59.835781 2666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:50:59.835795 2666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:59.845355 2666 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.130:39573
I20260810 18:50:59.845387 2812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.130:39573 every 8 connection(s)
I20260810 18:50:59.846267 2666 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
I20260810 18:50:59.855472 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 2666
I20260810 18:50:59.855588 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance
I20260810 18:50:59.855871 19106 external_mini_cluster.cc:1428] Setting key 3a05cb22d7c3f6da3db505c6427e9989
Aug 10 18:50:59 dist-test-slave-cbsf krb5kdc[2388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387859, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:50:59.861037 2485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:59.848100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:40833 (local address 127.18.168.190:40999)
0810 18:50:59.848238 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:59.848242 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:59.849108 (+ 866us) server_negotiation.cc:407] Connection header received
0810 18:50:59.850515 (+ 1407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:59.850518 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:59.850574 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:59.850692 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:50:59.851868 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:59.852449 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:59.853664 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:59.853888 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:59.853971 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:59.856795 (+ 2824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:50:59.856799 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:50:59.856802 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:50:59.856833 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:50:59.858651 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:59.859291 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:59.859297 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:59.859302 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:59.859399 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:59.859742 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:59.859748 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:59.859753 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:59.860016 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:50:59.860122 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:59.860823 (+ 701us) server_negotiation.cc:299] Negotiation successful
0810 18:50:59.860922 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 18:50:59.862160 2816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:59.848460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.130:40833)
0810 18:50:59.848962 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:59.849003 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 18:50:59.850295 (+ 1292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:50:59.850868 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:50:59.850875 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:50:59.851219 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:50:59.851711 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:59.851724 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:59.852588 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:59.852591 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:59.853544 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:59.853554 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:59.854033 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:59.854035 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:59.854178 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:59.854805 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:50:59.854821 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:50:59.856556 (+ 1735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:50:59.858767 (+ 2211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:59.858794 (+ 27us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:59.858810 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:59.859192 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:59.859499 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:59.859501 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:59.859502 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:59.859644 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:59.860147 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:50:59.860155 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:50:59.860481 (+ 326us) client_negotiation.cc:769] Sending connection context
0810 18:50:59.860763 (+ 282us) client_negotiation.cc:240] Negotiation successful
0810 18:50:59.861102 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":184,"threads_started":1}
I20260810 18:50:59.863365 2814 heartbeater.cc:347] Connected to a master server at 127.18.168.190:40999
I20260810 18:50:59.863770 2814 heartbeater.cc:464] Registering TS with master...
I20260810 18:50:59.864311 2814 heartbeater.cc:511] Master 127.18.168.190:40999 requested a full tablet report, sending...
I20260810 18:50:59.865799 2440 ts_manager.cc:194] Registered new tserver with Master: 436302659835484193079dea06d4e462 (127.18.168.130:39573)
I20260810 18:50:59.866427 2440 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.130@KRBTEST.COM'} at 127.18.168.130:40833
WARNING: no policy specified for kudu/127.18.168.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab.
Entry for principal kudu/127.18.168.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab.
I20260810 18:50:59.888163 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.131:0
--local_ip_for_outbound_sockets=127.18.168.131
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:40999
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:39893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:51:00.005290 2819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:51:00.005584 2819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:51:00.005669 2819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:51:00.009676 2819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:51:00.009753 2819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:51:00.009837 2819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.131
I20260810 18:51:00.015439 2819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:39893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:40999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.131
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:51:00.016611 2819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:51:00.017692 2819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:51:00.027725 2825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:00.027791 2824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:00.027793 2827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:00.027865 2819 server_base.cc:1034] running on GCE node
I20260810 18:51:00.028309 2819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:51:00.029052 2819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:51:00.030278 2819 hybrid_clock.cc:648] HybridClock initialized: now 1786387860030239 us; error 54 us; skew 500 ppm
Aug 10 18:51:00 dist-test-slave-cbsf krb5kdc[2388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387860, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:00.033119 2819 init.cc:377] Logged in from keytab as kudu/127.18.168.131@KRBTEST.COM (short username kudu)
I20260810 18:51:00.033953 2819 webserver.cc:460] Webserver started at http://127.18.168.131:42871/ using document root <none> and password file <none>
I20260810 18:51:00.034518 2819 fs_manager.cc:359] Metadata directory not provided
I20260810 18:51:00.034615 2819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:51:00.034854 2819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:51:00.036938 2819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/data/instance:
uuid: "a6bbcf658b984e70bdc8f1826d4f8222"
format_stamp: "Formatted at 2026-08-10 18:51:00 on dist-test-slave-cbsf"
server_key: "a4250aa6aa5c15e39fbd8d699a003537"
server_key_iv: "264b2e4677ec8a84a2e3c4e964c90f3b"
server_key_version: "encryptionkey@0"
I20260810 18:51:00.037539 2819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance:
uuid: "a6bbcf658b984e70bdc8f1826d4f8222"
format_stamp: "Formatted at 2026-08-10 18:51:00 on dist-test-slave-cbsf"
server_key: "a4250aa6aa5c15e39fbd8d699a003537"
server_key_iv: "264b2e4677ec8a84a2e3c4e964c90f3b"
server_key_version: "encryptionkey@0"
I20260810 18:51:00.041172 2819 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:51:00.043553 2834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:00.044786 2819 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:51:00.044915 2819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/wal
uuid: "a6bbcf658b984e70bdc8f1826d4f8222"
format_stamp: "Formatted at 2026-08-10 18:51:00 on dist-test-slave-cbsf"
server_key: "a4250aa6aa5c15e39fbd8d699a003537"
server_key_iv: "264b2e4677ec8a84a2e3c4e964c90f3b"
server_key_version: "encryptionkey@0"
I20260810 18:51:00.045025 2819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:51:00.061000 2819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:51:00.063859 2819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:51:00.064079 2819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:51:00.064776 2819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:51:00.065626 2819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:51:00.065713 2819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:00.065781 2819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:51:00.065815 2819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:00.075227 2819 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.131:38147
I20260810 18:51:00.075269 2967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.131:38147 every 8 connection(s)
I20260810 18:51:00.076124 2819 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
I20260810 18:51:00.085527 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 2819
I20260810 18:51:00.085630 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance
Aug 10 18:51:00 dist-test-slave-cbsf krb5kdc[2388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387860, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:51:00.085947 19106 external_mini_cluster.cc:1428] Setting key 8e0f208c80763fc9b597a743b02a1f1d
I20260810 18:51:00.090267 2485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:00.077973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:43029 (local address 127.18.168.190:40999)
0810 18:51:00.078127 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:00.078131 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:00.079114 (+ 983us) server_negotiation.cc:407] Connection header received
0810 18:51:00.080461 (+ 1347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:00.080464 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:00.080513 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:00.080589 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:00.081779 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:00.082309 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:00.083403 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:00.083616 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:00.083691 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:00.086501 (+ 2810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:00.086508 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:00.086511 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:00.086545 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:00.088141 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:00.088669 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:00.088672 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:00.088674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:00.088794 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:00.089162 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:00.089165 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:00.089167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:00.089356 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:00.089473 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:00.089966 (+ 493us) server_negotiation.cc:299] Negotiation successful
0810 18:51:00.090085 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 18:51:00.091133 2972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:00.078411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.131:43029)
0810 18:51:00.078957 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:00.079000 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:00.080267 (+ 1267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:00.080817 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:00.080824 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:00.081170 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:51:00.081647 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:00.081660 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:00.082420 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:00.082422 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:00.083297 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:00.083306 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:00.083748 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:00.083751 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:00.083940 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:00.084563 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:51:00.084580 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:51:00.086335 (+ 1755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:51:00.088271 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:00.088277 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:00.088290 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:00.088551 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:00.088897 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:00.088900 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:00.088902 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:00.089050 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:00.089474 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:51:00.089481 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:51:00.089730 (+ 249us) client_negotiation.cc:769] Sending connection context
0810 18:51:00.089924 (+ 194us) client_negotiation.cc:240] Negotiation successful
0810 18:51:00.090241 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":124,"threads_started":1}
I20260810 18:51:00.092361 2969 heartbeater.cc:347] Connected to a master server at 127.18.168.190:40999
I20260810 18:51:00.092618 2969 heartbeater.cc:464] Registering TS with master...
I20260810 18:51:00.093132 2969 heartbeater.cc:511] Master 127.18.168.190:40999 requested a full tablet report, sending...
I20260810 18:51:00.094126 2440 ts_manager.cc:194] Registered new tserver with Master: a6bbcf658b984e70bdc8f1826d4f8222 (127.18.168.131:38147)
I20260810 18:51:00.094645 2440 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.131@KRBTEST.COM'} at 127.18.168.131:43029
I20260810 18:51:00.099861 19106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:51:00.109056 2485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:00.101939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39022 (local address 127.18.168.190:40999)
0810 18:51:00.102059 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:00.102064 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:00.102295 (+ 231us) server_negotiation.cc:407] Connection header received
0810 18:51:00.102377 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:00.102379 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:00.102413 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:00.102480 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:00.103164 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:00.103668 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:00.104481 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:00.104736 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:00.104836 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:00.106105 (+ 1269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:00.106108 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:00.106109 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:00.106130 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:00.107592 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:00.107965 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:00.107968 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:00.107969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:00.108041 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:00.108307 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:00.108309 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:00.108310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:00.108509 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:00.108624 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:00.108856 (+ 232us) server_negotiation.cc:299] Negotiation successful
0810 18:51:00.108945 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 18:51:00.111967 2440 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39022:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:51:00.115197 2440 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:51:00.129760 2981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:00.123609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:37577 (local address 127.0.0.1:60888)
0810 18:51:00.123955 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:00.124008 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:00.124156 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:00.125622 (+ 1466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:00.125628 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:00.125651 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:51:00.125797 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:00.125811 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:00.126945 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:00.126950 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:00.128171 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:00.128180 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:00.129215 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:00.129218 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:00.129343 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:00.129389 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 18:51:00.129501 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 18:51:00.129610 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":32,"thread_start_us":82,"threads_started":1}
I20260810 18:51:00.129810 2982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:00.123768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60888 (local address 127.18.168.129:37577)
0810 18:51:00.124646 (+ 878us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:00.124665 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:00.124711 (+ 46us) server_negotiation.cc:407] Connection header received
0810 18:51:00.124796 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:00.124804 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:00.125439 (+ 635us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:00.125628 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:51:00.125993 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:00.126799 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:00.128309 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:00.129069 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:00.129232 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:00.129361 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:00.129552 (+ 191us) server_negotiation.cc:299] Negotiation successful
0810 18:51:00.129652 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":773,"thread_start_us":81,"threads_started":1}
I20260810 18:51:00.130378 2983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:00.124320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.131:38147 (local address 127.0.0.1:59228)
0810 18:51:00.125082 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:00.125095 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:00.125186 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:00.125888 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:00.125890 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:00.125911 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:51:00.126006 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:00.126012 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:00.127462 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:00.127465 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:00.128863 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:00.128874 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:00.130005 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:00.130011 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:00.130102 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:00.130133 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 18:51:00.130185 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 18:51:00.130254 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":705,"mutex_wait_us":44,"thread_start_us":548,"threads_started":1}
I20260810 18:51:00.130985 2984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:00.123835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:39573 (local address 127.0.0.1:44720)
0810 18:51:00.124605 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:00.124619 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:00.124734 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:00.125789 (+ 1055us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:00.125796 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:00.125818 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:51:00.125927 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:00.125934 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:00.127462 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:00.127465 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:00.129020 (+ 1555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:00.129034 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:00.130386 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:00.130390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:00.130660 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:00.130681 (+ 21us) client_negotiation.cc:769] Sending connection context
0810 18:51:00.130783 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 18:51:00.130854 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"mutex_wait_us":261,"thread_start_us":114,"threads_started":1}
I20260810 18:51:00.131080 2985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:00.124422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44720 (local address 127.18.168.130:39573)
0810 18:51:00.125067 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:00.125079 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:00.125105 (+ 26us) server_negotiation.cc:407] Connection header received
0810 18:51:00.125203 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:00.125210 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:00.125424 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:00.125589 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:51:00.126059 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:00.127290 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:00.129151 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:00.130242 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:00.130389 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:00.130523 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:00.130818 (+ 295us) server_negotiation.cc:299] Negotiation successful
0810 18:51:00.130911 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":108,"threads_started":1}
I20260810 18:51:00.132961 2585 tablet_service.cc:1467] Processing CreateTablet for tablet c25799f68c4a47fd9e03401727f7bc13 (DEFAULT_TABLE table=test-table [id=878b1efa174549869bc23b9cc8b0fc7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:51:00.133955 2585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c25799f68c4a47fd9e03401727f7bc13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:51:00.133713 2735 tablet_service.cc:1467] Processing CreateTablet for tablet c25799f68c4a47fd9e03401727f7bc13 (DEFAULT_TABLE table=test-table [id=878b1efa174549869bc23b9cc8b0fc7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:51:00.134589 2735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c25799f68c4a47fd9e03401727f7bc13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:51:00.135186 2986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:00.124949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59228 (local address 127.18.168.131:38147)
0810 18:51:00.125439 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:00.125451 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:00.125473 (+ 22us) server_negotiation.cc:407] Connection header received
0810 18:51:00.125551 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:00.125556 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:00.125738 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:00.126034 (+ 296us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:51:00.126194 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:00.127297 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:00.128995 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:00.129858 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:00.130669 (+ 811us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:00.134889 (+ 4220us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:00.134945 (+ 56us) server_negotiation.cc:299] Negotiation successful
0810 18:51:00.135042 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":146,"threads_started":1}
I20260810 18:51:00.138221 2888 tablet_service.cc:1467] Processing CreateTablet for tablet c25799f68c4a47fd9e03401727f7bc13 (DEFAULT_TABLE table=test-table [id=878b1efa174549869bc23b9cc8b0fc7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:51:00.139261 2888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c25799f68c4a47fd9e03401727f7bc13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:51:00.140931 2987 tablet_bootstrap.cc:492] T c25799f68c4a47fd9e03401727f7bc13 P 7c3c35e869e641a98116d03ab65f70ca: Bootstrap starting.
I20260810 18:51:00.143211 2987 tablet_bootstrap.cc:654] T c25799f68c4a47fd9e03401727f7bc13 P 7c3c35e869e641a98116d03ab65f70ca: Neither blocks nor log segments found. Creating new log.
I20260810 18:51:00.143956 2987 log.cc:824] T c25799f68c4a47fd9e03401727f7bc13 P 7c3c35e869e641a98116d03ab65f70ca: Log is configured to *not* fsync() on all Append() calls
I20260810 18:51:00.145820 2991 tablet_bootstrap.cc:492] T c25799f68c4a47fd9e03401727f7bc13 P a6bbcf658b984e70bdc8f1826d4f8222: Bootstrap starting.
I20260810 18:51:00.146114 2987 tablet_bootstrap.cc:492] T c25799f68c4a47fd9e03401727f7bc13 P 7c3c35e869e641a98116d03ab65f70ca: No bootstrap required, opened a new log
I20260810 18:51:00.146313 2987 ts_tablet_manager.cc:1397] T c25799f68c4a47fd9e03401727f7bc13 P 7c3c35e869e641a98116d03ab65f70ca: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 18:51:00.146385 2989 tablet_bootstrap.cc:492] T c25799f68c4a47fd9e03401727f7bc13 P 436302659835484193079dea06d4e462: Bootstrap starting.
I20260810 18:51:00.148052 2991 tablet_bootstrap.cc:654] T c25799f68c4a47fd9e03401727f7bc13 P a6bbcf658b984e70bdc8f1826d4f8222: Neither blocks nor log segments found. Creating new log.
I20260810 18:51:00.149088 2989 tablet_bootstrap.cc:654] T c25799f68c4a47fd9e03401727f7bc13 P 436302659835484193079dea06d4e462: Neither blocks nor log segments found. Creating new log.
I20260810 18:51:00.149530 2991 log.cc:824] T c25799f68c4a47fd9e03401727f7bc13 P a6bbcf658b984e70bdc8f1826d4f8222: Log is configured to *not* fsync() on all Append() calls
I20260810 18:51:00.149916 2989 log.cc:824] T c25799f68c4a47fd9e03401727f7bc13 P 436302659835484193079dea06d4e462: Log is configured to *not* fsync() on all Append() calls
I20260810 18:51:00.150120 2987 raft_consensus.cc:348] T c25799f68c4a47fd9e03401727f7bc13 P 7c3c35e869e641a98116d03ab65f70ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c3c35e869e641a98116d03ab65f70ca" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37577 } } peers { permanent_uuid: "a6bbcf658b984e70bdc8f1826d4f8222" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 38147 } } peers { permanent_uuid: "436302659835484193079dea06d4e462" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39573 } }
I20260810 18:51:00.150460 2987 raft_consensus.cc:374] T c25799f68c4a47fd9e03401727f7bc13 P 7c3c35e869e641a98116d03ab65f70ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:51:00.150517 2987 raft_consensus.cc:729] T c25799f68c4a47fd9e03401727f7bc13 P 7c3c35e869e641a98116d03ab65f70ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c3c35e869e641a98116d03ab65f70ca, State: Initialized, Role: FOLLOWER
I20260810 18:51:00.150938 2987 consensus_queue.cc:260] T c25799f68c4a47fd9e03401727f7bc13 P 7c3c35e869e641a98116d03ab65f70ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c3c35e869e641a98116d03ab65f70ca" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37577 } } peers { permanent_uuid: "a6bbcf658b984e70bdc8f1826d4f8222" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 38147 } } peers { permanent_uuid: "436302659835484193079dea06d4e462" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39573 } }
I20260810 18:51:00.151568 2991 tablet_bootstrap.cc:492] T c25799f68c4a47fd9e03401727f7bc13 P a6bbcf658b984e70bdc8f1826d4f8222: No bootstrap required, opened a new log
I20260810 18:51:00.151647 2660 heartbeater.cc:503] Master 127.18.168.190:40999 was elected leader, sending a full tablet report...
I20260810 18:51:00.151803 2989 tablet_bootstrap.cc:492] T c25799f68c4a47fd9e03401727f7bc13 P 436302659835484193079dea06d4e462: No bootstrap required, opened a new log
I20260810 18:51:00.151958 2989 ts_tablet_manager.cc:1397] T c25799f68c4a47fd9e03401727f7bc13 P 436302659835484193079dea06d4e462: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 18:51:00.151970 2987 ts_tablet_manager.cc:1428] T c25799f68c4a47fd9e03401727f7bc13 P 7c3c35e869e641a98116d03ab65f70ca: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 18:51:00.152305 2991 ts_tablet_manager.cc:1397] T c25799f68c4a47fd9e03401727f7bc13 P a6bbcf658b984e70bdc8f1826d4f8222: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.006s
I20260810 18:51:00.155051 2989 raft_consensus.cc:348] T c25799f68c4a47fd9e03401727f7bc13 P 436302659835484193079dea06d4e462 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c3c35e869e641a98116d03ab65f70ca" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37577 } } peers { permanent_uuid: "a6bbcf658b984e70bdc8f1826d4f8222" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 38147 } } peers { permanent_uuid: "436302659835484193079dea06d4e462" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39573 } }
I20260810 18:51:00.155314 2989 raft_consensus.cc:374] T c25799f68c4a47fd9e03401727f7bc13 P 436302659835484193079dea06d4e462 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:51:00.155376 2989 raft_consensus.cc:729] T c25799f68c4a47fd9e03401727f7bc13 P 436302659835484193079dea06d4e462 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 436302659835484193079dea06d4e462, State: Initialized, Role: FOLLOWER
I20260810 18:51:00.155783 2989 consensus_queue.cc:260] T c25799f68c4a47fd9e03401727f7bc13 P 436302659835484193079dea06d4e462 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c3c35e869e641a98116d03ab65f70ca" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37577 } } peers { permanent_uuid: "a6bbcf658b984e70bdc8f1826d4f8222" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 38147 } } peers { permanent_uuid: "436302659835484193079dea06d4e462" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39573 } }
I20260810 18:51:00.156457 2814 heartbeater.cc:503] Master 127.18.168.190:40999 was elected leader, sending a full tablet report...
I20260810 18:51:00.156435 2991 raft_consensus.cc:348] T c25799f68c4a47fd9e03401727f7bc13 P a6bbcf658b984e70bdc8f1826d4f8222 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c3c35e869e641a98116d03ab65f70ca" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37577 } } peers { permanent_uuid: "a6bbcf658b984e70bdc8f1826d4f8222" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 38147 } } peers { permanent_uuid: "436302659835484193079dea06d4e462" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39573 } }
I20260810 18:51:00.156731 2991 raft_consensus.cc:374] T c25799f68c4a47fd9e03401727f7bc13 P a6bbcf658b984e70bdc8f1826d4f8222 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:51:00.156805 2991 raft_consensus.cc:729] T c25799f68c4a47fd9e03401727f7bc13 P a6bbcf658b984e70bdc8f1826d4f8222 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6bbcf658b984e70bdc8f1826d4f8222, State: Initialized, Role: FOLLOWER
I20260810 18:51:00.157274 2991 consensus_queue.cc:260] T c25799f68c4a47fd9e03401727f7bc13 P a6bbcf658b984e70bdc8f1826d4f8222 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c3c35e869e641a98116d03ab65f70ca" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37577 } } peers { permanent_uuid: "a6bbcf658b984e70bdc8f1826d4f8222" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 38147 } } peers { permanent_uuid: "436302659835484193079dea06d4e462" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39573 } }
I20260810 18:51:00.157855 2989 ts_tablet_manager.cc:1428] T c25799f68c4a47fd9e03401727f7bc13 P 436302659835484193079dea06d4e462: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 18:51:00.158325 2991 ts_tablet_manager.cc:1428] T c25799f68c4a47fd9e03401727f7bc13 P a6bbcf658b984e70bdc8f1826d4f8222: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260810 18:51:00.158382 2969 heartbeater.cc:503] Master 127.18.168.190:40999 was elected leader, sending a full tablet report...
I20260810 18:51:00.188546 2994 raft_consensus.cc:482] T c25799f68c4a47fd9e03401727f7bc13 P 7c3c35e869e641a98116d03ab65f70ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:51:00.188766 2994 raft_consensus.cc:504] T c25799f68c4a47fd9e03401727f7bc13 P 7c3c35e869e641a98116d03ab65f70ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c3c35e869e641a98116d03ab65f70ca" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37577 } } peers { permanent_uuid: "a6bbcf658b984e70bdc8f1826d4f8222" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 38147 } } peers { permanent_uuid: "436302659835484193079dea06d4e462" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39573 } }
I20260810 18:51:00.189893 2994 leader_election.cc:288] T c25799f68c4a47fd9e03401727f7bc13 P 7c3c35e869e641a98116d03ab65f70ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6bbcf658b984e70bdc8f1826d4f8222 (127.18.168.131:38147), 436302659835484193079dea06d4e462 (127.18.168.130:39573)
I20260810 18:51:00.194033 2985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:00.190491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:58775 (local address 127.18.168.130:39573)
0810 18:51:00.190662 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:00.190667 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:00.190693 (+ 26us) server_negotiation.cc:407] Connection header received
0810 18:51:00.190759 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:00.190761 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:00.190829 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:00.190898 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:51:00.191254 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:00.191719 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:00.193036 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:00.193438 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:00.193593 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:00.193651 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:00.193834 (+ 183us) server_negotiation.cc:299] Negotiation successful
0810 18:51:00.193901 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 18:51:00.194056 2986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:00.190221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:38799 (local address 127.18.168.131:38147)
0810 18:51:00.190366 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:00.190372 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:00.190520 (+ 148us) server_negotiation.cc:407] Connection header received
0810 18:51:00.190691 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:00.190697 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:00.190757 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:00.190831 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:51:00.191195 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:00.191714 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:00.193009 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:00.193433 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:00.193594 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:00.193648 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:00.193876 (+ 228us) server_negotiation.cc:299] Negotiation successful
0810 18:51:00.193951 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 18:51:00.194067 2997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:00.190080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.131:38147 (local address 127.18.168.129:38799)
0810 18:51:00.190422 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:00.190438 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:00.190557 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:00.190860 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:00.190865 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:00.190886 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:51:00.191066 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:00.191072 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:00.191832 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:00.191838 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:00.192881 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:00.192891 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:00.193593 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:00.193597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:00.193719 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:00.193737 (+ 18us) client_negotiation.cc:769] Sending connection context
0810 18:51:00.193834 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 18:51:00.193903 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":76,"thread_start_us":141,"threads_started":1}
I20260810 18:51:00.194082 2998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:00.190179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:39573 (local address 127.18.168.129:58775)
0810 18:51:00.190528 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:00.190542 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:00.190657 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:00.190941 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:00.190945 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:00.190962 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:51:00.191095 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:00.191099 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:00.191841 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:00.191845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:00.192891 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:00.192901 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:00.193589 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:00.193593 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:00.193691 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:00.193716 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 18:51:00.193789 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 18:51:00.193887 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":62,"thread_start_us":157,"threads_started":1}
I20260810 18:51:00.194988 2915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c25799f68c4a47fd9e03401727f7bc13" candidate_uuid: "7c3c35e869e641a98116d03ab65f70ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6bbcf658b984e70bdc8f1826d4f8222" is_pre_election: true
I20260810 18:51:00.194998 2758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c25799f68c4a47fd9e03401727f7bc13" candidate_uuid: "7c3c35e869e641a98116d03ab65f70ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "436302659835484193079dea06d4e462" is_pre_election: true
I20260810 18:51:00.195293 2758 raft_consensus.cc:2454] T c25799f68c4a47fd9e03401727f7bc13 P 436302659835484193079dea06d4e462 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c3c35e869e641a98116d03ab65f70ca in term 0.
I20260810 18:51:00.195297 2915 raft_consensus.cc:2454] T c25799f68c4a47fd9e03401727f7bc13 P a6bbcf658b984e70bdc8f1826d4f8222 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c3c35e869e641a98116d03ab65f70ca in term 0.
I20260810 18:51:00.195879 2533 leader_election.cc:302] T c25799f68c4a47fd9e03401727f7bc13 P 7c3c35e869e641a98116d03ab65f70ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 436302659835484193079dea06d4e462, 7c3c35e869e641a98116d03ab65f70ca; no voters:
I20260810 18:51:00.196211 2994 raft_consensus.cc:2781] T c25799f68c4a47fd9e03401727f7bc13 P 7c3c35e869e641a98116d03ab65f70ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:51:00.196302 2994 raft_consensus.cc:482] T c25799f68c4a47fd9e03401727f7bc13 P 7c3c35e869e641a98116d03ab65f70ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:51:00.196347 2994 raft_consensus.cc:3037] T c25799f68c4a47fd9e03401727f7bc13 P 7c3c35e869e641a98116d03ab65f70ca [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:51:00.197530 2994 raft_consensus.cc:504] T c25799f68c4a47fd9e03401727f7bc13 P 7c3c35e869e641a98116d03ab65f70ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c3c35e869e641a98116d03ab65f70ca" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37577 } } peers { permanent_uuid: "a6bbcf658b984e70bdc8f1826d4f8222" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 38147 } } peers { permanent_uuid: "436302659835484193079dea06d4e462" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39573 } }
I20260810 18:51:00.197866 2994 leader_election.cc:288] T c25799f68c4a47fd9e03401727f7bc13 P 7c3c35e869e641a98116d03ab65f70ca [CANDIDATE]: Term 1 election: Requested vote from peers a6bbcf658b984e70bdc8f1826d4f8222 (127.18.168.131:38147), 436302659835484193079dea06d4e462 (127.18.168.130:39573)
I20260810 18:51:00.198180 2758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c25799f68c4a47fd9e03401727f7bc13" candidate_uuid: "7c3c35e869e641a98116d03ab65f70ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "436302659835484193079dea06d4e462"
I20260810 18:51:00.198203 2915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c25799f68c4a47fd9e03401727f7bc13" candidate_uuid: "7c3c35e869e641a98116d03ab65f70ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6bbcf658b984e70bdc8f1826d4f8222"
I20260810 18:51:00.198289 2758 raft_consensus.cc:3037] T c25799f68c4a47fd9e03401727f7bc13 P 436302659835484193079dea06d4e462 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:51:00.198328 2915 raft_consensus.cc:3037] T c25799f68c4a47fd9e03401727f7bc13 P a6bbcf658b984e70bdc8f1826d4f8222 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:51:00.199788 2758 raft_consensus.cc:2454] T c25799f68c4a47fd9e03401727f7bc13 P 436302659835484193079dea06d4e462 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c3c35e869e641a98116d03ab65f70ca in term 1.
I20260810 18:51:00.199888 2915 raft_consensus.cc:2454] T c25799f68c4a47fd9e03401727f7bc13 P a6bbcf658b984e70bdc8f1826d4f8222 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c3c35e869e641a98116d03ab65f70ca in term 1.
I20260810 18:51:00.200055 2533 leader_election.cc:302] T c25799f68c4a47fd9e03401727f7bc13 P 7c3c35e869e641a98116d03ab65f70ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 436302659835484193079dea06d4e462, 7c3c35e869e641a98116d03ab65f70ca; no voters:
I20260810 18:51:00.200242 2994 raft_consensus.cc:2781] T c25799f68c4a47fd9e03401727f7bc13 P 7c3c35e869e641a98116d03ab65f70ca [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:51:00.200465 2994 raft_consensus.cc:686] T c25799f68c4a47fd9e03401727f7bc13 P 7c3c35e869e641a98116d03ab65f70ca [term 1 LEADER]: Becoming Leader. State: Replica: 7c3c35e869e641a98116d03ab65f70ca, State: Running, Role: LEADER
I20260810 18:51:00.200862 2994 consensus_queue.cc:237] T c25799f68c4a47fd9e03401727f7bc13 P 7c3c35e869e641a98116d03ab65f70ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c3c35e869e641a98116d03ab65f70ca" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37577 } } peers { permanent_uuid: "a6bbcf658b984e70bdc8f1826d4f8222" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 38147 } } peers { permanent_uuid: "436302659835484193079dea06d4e462" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39573 } }
I20260810 18:51:00.203989 2439 catalog_manager.cc:5495] T c25799f68c4a47fd9e03401727f7bc13 P 7c3c35e869e641a98116d03ab65f70ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c3c35e869e641a98116d03ab65f70ca (127.18.168.129). New cstate: current_term: 1 leader_uuid: "7c3c35e869e641a98116d03ab65f70ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c3c35e869e641a98116d03ab65f70ca" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6bbcf658b984e70bdc8f1826d4f8222" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 38147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "436302659835484193079dea06d4e462" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39573 } health_report { overall_health: UNKNOWN } } }
I20260810 18:51:00.227051 2982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:00.223566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60892 (local address 127.18.168.129:37577)
0810 18:51:00.223723 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:00.223728 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:00.223743 (+ 15us) server_negotiation.cc:407] Connection header received
0810 18:51:00.223858 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:00.223861 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:00.223922 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:00.224045 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:51:00.224270 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:00.224923 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:00.225721 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:00.225918 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:00.226010 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:00.226274 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:51:00.226644 (+ 370us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:51:00.226729 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:00.226833 (+ 104us) server_negotiation.cc:299] Negotiation successful
0810 18:51:00.226877 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 18:51:00.232888 2915 raft_consensus.cc:1261] T c25799f68c4a47fd9e03401727f7bc13 P a6bbcf658b984e70bdc8f1826d4f8222 [term 1 FOLLOWER]: Refusing update from remote peer 7c3c35e869e641a98116d03ab65f70ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:51:00.232887 2758 raft_consensus.cc:1261] T c25799f68c4a47fd9e03401727f7bc13 P 436302659835484193079dea06d4e462 [term 1 FOLLOWER]: Refusing update from remote peer 7c3c35e869e641a98116d03ab65f70ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:51:00.233664 2999 consensus_queue.cc:1035] T c25799f68c4a47fd9e03401727f7bc13 P 7c3c35e869e641a98116d03ab65f70ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "436302659835484193079dea06d4e462" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:51:00.233839 2994 consensus_queue.cc:1035] T c25799f68c4a47fd9e03401727f7bc13 P 7c3c35e869e641a98116d03ab65f70ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6bbcf658b984e70bdc8f1826d4f8222" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 38147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:51:00.242702 3003 mvcc.cc:205] Tried to move back new op lower bound from 7317044675506696192 to 7317044675385409536. Current Snapshot: MvccSnapshot[applied={T|T < 7317044675506696192}]
I20260810 18:51:00.250161 2440 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:51:00.250357 2440 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:51:00.254701 2440 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2: Sending DeleteTablet for 3 replicas of tablet c25799f68c4a47fd9e03401727f7bc13
I20260810 18:51:00.255703 2735 tablet_service.cc:1514] Processing DeleteTablet for tablet c25799f68c4a47fd9e03401727f7bc13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:51:00 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:44720
I20260810 18:51:00.255842 2585 tablet_service.cc:1514] Processing DeleteTablet for tablet c25799f68c4a47fd9e03401727f7bc13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:51:00 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:60888
I20260810 18:51:00.257018 2888 tablet_service.cc:1514] Processing DeleteTablet for tablet c25799f68c4a47fd9e03401727f7bc13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:51:00 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:59228
I20260810 18:51:00.257398 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 2513
I20260810 18:51:00.257618 3012 tablet_replica.cc:330] T c25799f68c4a47fd9e03401727f7bc13 P 436302659835484193079dea06d4e462: stopping tablet replica
I20260810 18:51:00.258046 3012 raft_consensus.cc:2229] T c25799f68c4a47fd9e03401727f7bc13 P 436302659835484193079dea06d4e462 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:51:00.258435 3012 raft_consensus.cc:2258] T c25799f68c4a47fd9e03401727f7bc13 P 436302659835484193079dea06d4e462 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:51:00.263183 3012 ts_tablet_manager.cc:1905] T c25799f68c4a47fd9e03401727f7bc13 P 436302659835484193079dea06d4e462: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:51:00.265362 3015 tablet_replica.cc:330] T c25799f68c4a47fd9e03401727f7bc13 P a6bbcf658b984e70bdc8f1826d4f8222: stopping tablet replica
I20260810 18:51:00.265713 3012 ts_tablet_manager.cc:1918] T c25799f68c4a47fd9e03401727f7bc13 P 436302659835484193079dea06d4e462: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:51:00.265726 3015 raft_consensus.cc:2229] T c25799f68c4a47fd9e03401727f7bc13 P a6bbcf658b984e70bdc8f1826d4f8222 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:51:00.265828 3012 log.cc:1196] T c25799f68c4a47fd9e03401727f7bc13 P 436302659835484193079dea06d4e462: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/wal/wals/c25799f68c4a47fd9e03401727f7bc13
I20260810 18:51:00.266078 3015 raft_consensus.cc:2258] T c25799f68c4a47fd9e03401727f7bc13 P a6bbcf658b984e70bdc8f1826d4f8222 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:51:00.266206 3012 ts_tablet_manager.cc:1939] T c25799f68c4a47fd9e03401727f7bc13 P 436302659835484193079dea06d4e462: Deleting consensus metadata
I20260810 18:51:00.267364 2427 catalog_manager.cc:4841] TS 436302659835484193079dea06d4e462 (127.18.168.130:39573): tablet c25799f68c4a47fd9e03401727f7bc13 (table test-table [id=878b1efa174549869bc23b9cc8b0fc7b]) successfully deleted
I20260810 18:51:00.267658 3015 ts_tablet_manager.cc:1905] T c25799f68c4a47fd9e03401727f7bc13 P a6bbcf658b984e70bdc8f1826d4f8222: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:51:00.271107 3015 ts_tablet_manager.cc:1918] T c25799f68c4a47fd9e03401727f7bc13 P a6bbcf658b984e70bdc8f1826d4f8222: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:51:00.271217 3015 log.cc:1196] T c25799f68c4a47fd9e03401727f7bc13 P a6bbcf658b984e70bdc8f1826d4f8222: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/wal/wals/c25799f68c4a47fd9e03401727f7bc13
I20260810 18:51:00.271574 3015 ts_tablet_manager.cc:1939] T c25799f68c4a47fd9e03401727f7bc13 P a6bbcf658b984e70bdc8f1826d4f8222: Deleting consensus metadata
W20260810 18:51:00.272128 2424 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.168.129:37577 (error 108)
I20260810 18:51:00.272933 2424 catalog_manager.cc:4841] TS a6bbcf658b984e70bdc8f1826d4f8222 (127.18.168.131:38147): tablet c25799f68c4a47fd9e03401727f7bc13 (table test-table [id=878b1efa174549869bc23b9cc8b0fc7b]) successfully deleted
I20260810 18:51:00.273191 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 2666
I20260810 18:51:00.276278 2984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:00.275820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:37577 (local address 127.0.0.1:60908)
0810 18:51:00.276082 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:00.276160 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.129:37577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
W20260810 18:51:00.276480 2424 catalog_manager.cc:4584] TS 7c3c35e869e641a98116d03ab65f70ca (127.18.168.129:37577): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c25799f68c4a47fd9e03401727f7bc13: Network error: Client connection negotiation failed: client connection to 127.18.168.129:37577: connect: Connection refused (error 111)
I20260810 18:51:00.283564 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 2819
I20260810 18:51:00.292462 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 2404
I20260810 18:51:00.305073 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:40999
--webserver_interface=127.18.168.190
--webserver_port=40807
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:39893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:40999
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:51:00.424050 3017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:51:00.424371 3017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:51:00.424414 3017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:51:00.428563 3017 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:51:00.428613 3017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:51:00.428632 3017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:51:00.428649 3017 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:51:00.428730 3017 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:51:00.434015 3017 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:39893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:40999
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:40999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=40807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:51:00.435115 3017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:51:00.436134 3017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:51:00.451792 3026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:00.451785 3024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:00.451813 3023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:00.451869 3017 server_base.cc:1034] running on GCE node
I20260810 18:51:00.452795 3017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:51:00.453987 3017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:51:00.455199 3017 hybrid_clock.cc:648] HybridClock initialized: now 1786387860455166 us; error 55 us; skew 500 ppm
Aug 10 18:51:00 dist-test-slave-cbsf krb5kdc[2388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387860, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:00.458158 3017 init.cc:377] Logged in from keytab as kudu/127.18.168.190@KRBTEST.COM (short username kudu)
I20260810 18:51:00.459177 3017 webserver.cc:460] Webserver started at http://127.18.168.190:40807/ using document root <none> and password file <none>
I20260810 18:51:00.459787 3017 fs_manager.cc:359] Metadata directory not provided
I20260810 18:51:00.459913 3017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:51:00.463892 3017 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:51:00.466128 3033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:00.467392 3017 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:51:00.467555 3017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "69089a0bf06544d4bbca5fc069539ac2"
format_stamp: "Formatted at 2026-08-10 18:50:59 on dist-test-slave-cbsf"
server_key: "9d15b64b5cca04f3ec706f523022d8e1"
server_key_iv: "da562ea5a3130604866f91c09367e839"
server_key_version: "encryptionkey@0"
I20260810 18:51:00.468196 3017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:51:00.484784 3017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:51:00.487491 3017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:51:00.487705 3017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:51:00.495349 3017 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:40999
I20260810 18:51:00.495635 3096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:40999 every 8 connection(s)
I20260810 18:51:00.496481 3017 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:51:00.501173 3097 sys_catalog.cc:263] Verifying existing consensus state
I20260810 18:51:00.502815 3097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2: Bootstrap starting.
I20260810 18:51:00.504369 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 3017
I20260810 18:51:00.504904 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.129:37577
--local_ip_for_outbound_sockets=127.18.168.129
--tserver_master_addrs=127.18.168.190:40999
--webserver_port=41297
--webserver_interface=127.18.168.129
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:39893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:51:00.522492 3097 log.cc:824] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2: Log is configured to *not* fsync() on all Append() calls
I20260810 18:51:00.531754 3097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:51:00.532192 3097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2: Bootstrap complete.
I20260810 18:51:00.536453 3097 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69089a0bf06544d4bbca5fc069539ac2" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 40999 } }
I20260810 18:51:00.536903 3097 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69089a0bf06544d4bbca5fc069539ac2, State: Initialized, Role: FOLLOWER
I20260810 18:51:00.537458 3097 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69089a0bf06544d4bbca5fc069539ac2" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 40999 } }
I20260810 18:51:00.537583 3097 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:51:00.537688 3097 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:51:00.537814 3097 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [term 1 FOLLOWER]: Advancing to term 2
I20260810 18:51:00.539772 3097 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69089a0bf06544d4bbca5fc069539ac2" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 40999 } }
I20260810 18:51:00.540143 3097 leader_election.cc:302] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69089a0bf06544d4bbca5fc069539ac2; no voters:
I20260810 18:51:00.540515 3097 leader_election.cc:288] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 18:51:00.540652 3101 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 18:51:00.540942 3101 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [term 2 LEADER]: Becoming Leader. State: Replica: 69089a0bf06544d4bbca5fc069539ac2, State: Running, Role: LEADER
I20260810 18:51:00.541276 3101 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69089a0bf06544d4bbca5fc069539ac2" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 40999 } }
I20260810 18:51:00.541796 3097 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:51:00.542681 3102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "69089a0bf06544d4bbca5fc069539ac2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69089a0bf06544d4bbca5fc069539ac2" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 40999 } } }
I20260810 18:51:00.542825 3102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [sys.catalog]: This master's current role is: LEADER
I20260810 18:51:00.543219 3103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69089a0bf06544d4bbca5fc069539ac2. Latest consensus state: current_term: 2 leader_uuid: "69089a0bf06544d4bbca5fc069539ac2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69089a0bf06544d4bbca5fc069539ac2" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 40999 } } }
I20260810 18:51:00.543370 3103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [sys.catalog]: This master's current role is: LEADER
I20260810 18:51:00.543726 3111 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:51:00.548434 3111 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:51:00.549187 3111 catalog_manager.cc:1259] Loaded cluster ID: 9dc514d0f3ce4c8da0755be8b4a96313
I20260810 18:51:00.549252 3111 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:51:00.550725 3111 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260810 18:51:00.555563 3111 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260810 18:51:00.560967 3111 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:51:00.561668 3111 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2: Loaded TSK: 0
I20260810 18:51:00.562858 3111 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 18:51:00.623102 3099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:51:00.623375 3099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:51:00.623450 3099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:51:00.627578 3099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:51:00.627658 3099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:51:00.627740 3099 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.129
I20260810 18:51:00.633119 3099 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:39893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.129:37577
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.168.129
--webserver_port=41297
--tserver_master_addrs=127.18.168.190:40999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3099
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.129
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:51:00.634123 3099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:51:00.635314 3099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:51:00.646728 3127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:00.646728 3130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:00.646926 3099 server_base.cc:1034] running on GCE node
W20260810 18:51:00.646728 3128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:00.647591 3099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:51:00.648212 3099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:51:00.649411 3099 hybrid_clock.cc:648] HybridClock initialized: now 1786387860649391 us; error 40 us; skew 500 ppm
Aug 10 18:51:00 dist-test-slave-cbsf krb5kdc[2388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387860, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:00.652493 3099 init.cc:377] Logged in from keytab as kudu/127.18.168.129@KRBTEST.COM (short username kudu)
I20260810 18:51:00.653672 3099 webserver.cc:460] Webserver started at http://127.18.168.129:41297/ using document root <none> and password file <none>
I20260810 18:51:00.654386 3099 fs_manager.cc:359] Metadata directory not provided
I20260810 18:51:00.654502 3099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:51:00.659607 3099 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260810 18:51:00.662251 3137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:00.663806 3099 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 18:51:00.663937 3099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/wal
uuid: "7c3c35e869e641a98116d03ab65f70ca"
format_stamp: "Formatted at 2026-08-10 18:50:59 on dist-test-slave-cbsf"
server_key: "2cf66e56f2f034576bd6dd15490fd60e"
server_key_iv: "c32eb7f3ba0919eb3ffb84505d19d9fe"
server_key_version: "encryptionkey@0"
I20260810 18:51:00.664618 3099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:51:00.694496 3099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:51:00.697923 3099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:51:00.698161 3099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:51:00.698807 3099 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:51:00.699923 3144 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 18:51:00.704423 3099 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 18:51:00.704506 3099 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260810 18:51:00.704612 3099 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 18:51:00.706301 3099 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 18:51:00.706378 3099 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260810 18:51:00.706693 3144 tablet_bootstrap.cc:492] T c25799f68c4a47fd9e03401727f7bc13 P 7c3c35e869e641a98116d03ab65f70ca: Bootstrap starting.
I20260810 18:51:00.716984 3099 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.129:37577
I20260810 18:51:00.717047 3271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.129:37577 every 8 connection(s)
I20260810 18:51:00.717929 3099 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
I20260810 18:51:00.722950 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 3099
I20260810 18:51:00.723513 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.130:39573
--local_ip_for_outbound_sockets=127.18.168.130
--tserver_master_addrs=127.18.168.190:40999
--webserver_port=39067
--webserver_interface=127.18.168.130
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:39893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:51:00.729466 3144 log.cc:824] T c25799f68c4a47fd9e03401727f7bc13 P 7c3c35e869e641a98116d03ab65f70ca: Log is configured to *not* fsync() on all Append() calls
Aug 10 18:51:00 dist-test-slave-cbsf krb5kdc[2388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387860, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:51:00.735828 3275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:00.720232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:51497 (local address 127.18.168.190:40999)
0810 18:51:00.720738 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:00.720758 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:00.720936 (+ 178us) server_negotiation.cc:407] Connection header received
0810 18:51:00.722359 (+ 1423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:00.722369 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:00.722639 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:00.722971 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:00.724340 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:00.725608 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:00.727420 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:00.727867 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:00.728050 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:00.730708 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:00.730714 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:00.730729 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:00.730763 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:00.732578 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:00.733162 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:00.733168 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:00.733173 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:00.733302 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:00.733674 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:00.733677 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:00.733678 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:00.734015 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:00.734169 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:00.734718 (+ 549us) server_negotiation.cc:299] Negotiation successful
0810 18:51:00.734981 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":110,"threads_started":1}
I20260810 18:51:00.735841 3274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:00.720063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.129:51497)
0810 18:51:00.720736 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:00.720800 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:00.721935 (+ 1135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:00.723021 (+ 1086us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:00.723031 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:00.723515 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:51:00.724148 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:00.724163 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:00.725833 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:00.725838 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:00.727276 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:00.727293 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:00.728031 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:00.728035 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:00.728228 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:00.728880 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:51:00.728896 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:51:00.730507 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:51:00.732737 (+ 2230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:00.732745 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:00.732759 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:00.733034 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:00.733410 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:00.733414 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:00.733416 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:00.733562 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:00.734154 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:51:00.734160 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:51:00.734393 (+ 233us) client_negotiation.cc:769] Sending connection context
0810 18:51:00.734590 (+ 197us) client_negotiation.cc:240] Negotiation successful
0810 18:51:00.734927 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":158,"threads_started":1}
I20260810 18:51:00.738267 3272 heartbeater.cc:347] Connected to a master server at 127.18.168.190:40999
I20260810 18:51:00.738543 3272 heartbeater.cc:464] Registering TS with master...
I20260810 18:51:00.739053 3272 heartbeater.cc:511] Master 127.18.168.190:40999 requested a full tablet report, sending...
I20260810 18:51:00.740545 3144 tablet_bootstrap.cc:492] T c25799f68c4a47fd9e03401727f7bc13 P 7c3c35e869e641a98116d03ab65f70ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:51:00.740964 3144 tablet_bootstrap.cc:492] T c25799f68c4a47fd9e03401727f7bc13 P 7c3c35e869e641a98116d03ab65f70ca: Bootstrap complete.
I20260810 18:51:00.741647 3060 ts_manager.cc:194] Registered new tserver with Master: 7c3c35e869e641a98116d03ab65f70ca (127.18.168.129:37577)
I20260810 18:51:00.742604 3144 ts_tablet_manager.cc:1397] T c25799f68c4a47fd9e03401727f7bc13 P 7c3c35e869e641a98116d03ab65f70ca: Time spent bootstrapping tablet: real 0.036s user 0.025s sys 0.007s
I20260810 18:51:00.745193 3060 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.129@KRBTEST.COM'} at 127.18.168.129:51497
I20260810 18:51:00.745479 3144 raft_consensus.cc:348] T c25799f68c4a47fd9e03401727f7bc13 P 7c3c35e869e641a98116d03ab65f70ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c3c35e869e641a98116d03ab65f70ca" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37577 } } peers { permanent_uuid: "a6bbcf658b984e70bdc8f1826d4f8222" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 38147 } } peers { permanent_uuid: "436302659835484193079dea06d4e462" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39573 } }
I20260810 18:51:00.746088 3144 raft_consensus.cc:729] T c25799f68c4a47fd9e03401727f7bc13 P 7c3c35e869e641a98116d03ab65f70ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c3c35e869e641a98116d03ab65f70ca, State: Initialized, Role: FOLLOWER
I20260810 18:51:00.746599 3144 consensus_queue.cc:260] T c25799f68c4a47fd9e03401727f7bc13 P 7c3c35e869e641a98116d03ab65f70ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c3c35e869e641a98116d03ab65f70ca" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37577 } } peers { permanent_uuid: "a6bbcf658b984e70bdc8f1826d4f8222" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 38147 } } peers { permanent_uuid: "436302659835484193079dea06d4e462" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39573 } }
I20260810 18:51:00.747555 3144 ts_tablet_manager.cc:1428] T c25799f68c4a47fd9e03401727f7bc13 P 7c3c35e869e641a98116d03ab65f70ca: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260810 18:51:00.747977 3272 heartbeater.cc:503] Master 127.18.168.190:40999 was elected leader, sending a full tablet report...
I20260810 18:51:00.751180 3278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:00.744475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:37577 (local address 127.0.0.1:34688)
0810 18:51:00.745880 (+ 1405us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:00.745922 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:00.746235 (+ 313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:00.747305 (+ 1070us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:00.747311 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:00.747339 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:51:00.747448 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:00.747454 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:00.748640 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:00.748643 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:00.749662 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:00.749673 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:00.750720 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:00.750724 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:00.750839 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:00.750882 (+ 43us) client_negotiation.cc:769] Sending connection context
0810 18:51:00.750983 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 18:51:00.751077 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1250,"thread_start_us":101,"threads_started":1}
I20260810 18:51:00.751260 3279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:00.745209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34688 (local address 127.18.168.129:37577)
0810 18:51:00.745643 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:00.745654 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:00.746204 (+ 550us) server_negotiation.cc:407] Connection header received
0810 18:51:00.746425 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:00.746430 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:00.747013 (+ 583us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:00.747166 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:51:00.747578 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:00.748541 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:00.749789 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:00.750597 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:00.750730 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:00.750864 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:00.751027 (+ 163us) server_negotiation.cc:299] Negotiation successful
0810 18:51:00.751113 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":102,"threads_started":1}
I20260810 18:51:00.752276 3194 tablet_service.cc:1514] Processing DeleteTablet for tablet c25799f68c4a47fd9e03401727f7bc13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:51:00 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:34688
I20260810 18:51:00.753090 3281 tablet_replica.cc:330] stopping tablet replica
I20260810 18:51:00.753372 3281 raft_consensus.cc:2229] T c25799f68c4a47fd9e03401727f7bc13 P 7c3c35e869e641a98116d03ab65f70ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:51:00.753624 3281 raft_consensus.cc:2258] T c25799f68c4a47fd9e03401727f7bc13 P 7c3c35e869e641a98116d03ab65f70ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:51:00.754521 3281 ts_tablet_manager.cc:1905] T c25799f68c4a47fd9e03401727f7bc13 P 7c3c35e869e641a98116d03ab65f70ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:51:00.758392 3281 ts_tablet_manager.cc:1918] T c25799f68c4a47fd9e03401727f7bc13 P 7c3c35e869e641a98116d03ab65f70ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:51:00.758514 3281 log.cc:1196] T c25799f68c4a47fd9e03401727f7bc13 P 7c3c35e869e641a98116d03ab65f70ca: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/wal/wals/c25799f68c4a47fd9e03401727f7bc13
I20260810 18:51:00.758771 3281 ts_tablet_manager.cc:1939] T c25799f68c4a47fd9e03401727f7bc13 P 7c3c35e869e641a98116d03ab65f70ca: Deleting consensus metadata
I20260810 18:51:00.760351 3036 catalog_manager.cc:4841] TS 7c3c35e869e641a98116d03ab65f70ca (127.18.168.129:37577): tablet c25799f68c4a47fd9e03401727f7bc13 (table test-table [id=878b1efa174549869bc23b9cc8b0fc7b]) successfully deleted
W20260810 18:51:00.848452 3276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:51:00.848793 3276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:51:00.848897 3276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:51:00.853102 3276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:51:00.853160 3276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:51:00.853268 3276 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.130
I20260810 18:51:00.858610 3276 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:39893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.130:39573
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.168.130
--webserver_port=39067
--tserver_master_addrs=127.18.168.190:40999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3276
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.130
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:51:00.859740 3276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:51:00.860909 3276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:51:00.871098 3286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:00.871109 3287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:00.871119 3292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:00.871481 3276 server_base.cc:1034] running on GCE node
I20260810 18:51:00.871886 3276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:51:00.872658 3276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:51:00.873900 3276 hybrid_clock.cc:648] HybridClock initialized: now 1786387860873860 us; error 50 us; skew 500 ppm
Aug 10 18:51:00 dist-test-slave-cbsf krb5kdc[2388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387860, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:00.876806 3276 init.cc:377] Logged in from keytab as kudu/127.18.168.130@KRBTEST.COM (short username kudu)
I20260810 18:51:00.877804 3276 webserver.cc:460] Webserver started at http://127.18.168.130:39067/ using document root <none> and password file <none>
I20260810 18:51:00.878429 3276 fs_manager.cc:359] Metadata directory not provided
I20260810 18:51:00.878541 3276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:51:00.882547 3276 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:51:00.884653 3299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:00.885939 3276 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:51:00.886075 3276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/wal
uuid: "436302659835484193079dea06d4e462"
format_stamp: "Formatted at 2026-08-10 18:50:59 on dist-test-slave-cbsf"
server_key: "102fe108fde9dcf0179f2fec6854b3a3"
server_key_iv: "9b39e195684ab7b07072f6f3d835150d"
server_key_version: "encryptionkey@0"
I20260810 18:51:00.886621 3276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:51:00.907620 3276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:51:00.910796 3276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:51:00.911023 3276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:51:00.911687 3276 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:51:00.912593 3276 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:51:00.912642 3276 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:00.912743 3276 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:51:00.912778 3276 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:00.922565 3276 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.130:39573
I20260810 18:51:00.922614 3433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.130:39573 every 8 connection(s)
I20260810 18:51:00.923681 3276 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
I20260810 18:51:00.933341 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 3276
I20260810 18:51:00.933817 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.131:38147
--local_ip_for_outbound_sockets=127.18.168.131
--tserver_master_addrs=127.18.168.190:40999
--webserver_port=42871
--webserver_interface=127.18.168.131
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:39893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 18:51:00 dist-test-slave-cbsf krb5kdc[2388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387860, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:51:00.938377 3275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:00.925844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:49413 (local address 127.18.168.190:40999)
0810 18:51:00.926002 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:00.926006 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:00.926898 (+ 892us) server_negotiation.cc:407] Connection header received
0810 18:51:00.928410 (+ 1512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:00.928413 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:00.928489 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:00.928605 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:00.929822 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:00.930365 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:00.931529 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:00.931764 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:00.931879 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:00.934612 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:00.934615 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:00.934617 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:00.934642 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:00.936157 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:00.936735 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:00.936741 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:00.936745 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:00.936837 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:00.937245 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:00.937249 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:00.937251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:00.937441 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:00.937562 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:00.938062 (+ 500us) server_negotiation.cc:299] Negotiation successful
0810 18:51:00.938201 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 18:51:00.939175 3437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:00.926216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.130:49413)
0810 18:51:00.926735 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:00.926792 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:00.928196 (+ 1404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:00.928806 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:00.928813 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:00.929179 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:51:00.929662 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:00.929676 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:00.930519 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:00.930522 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:00.931380 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:00.931390 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:00.931922 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:00.931927 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:00.932105 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:00.932879 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:51:00.932896 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:51:00.934418 (+ 1522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:51:00.936303 (+ 1885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:00.936308 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:00.936323 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:00.936592 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:00.936947 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:00.936950 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:00.936952 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:00.937102 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:00.937562 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:51:00.937570 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:51:00.937821 (+ 251us) client_negotiation.cc:769] Sending connection context
0810 18:51:00.938017 (+ 196us) client_negotiation.cc:240] Negotiation successful
0810 18:51:00.938345 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":116,"threads_started":1}
I20260810 18:51:00.940596 3435 heartbeater.cc:347] Connected to a master server at 127.18.168.190:40999
I20260810 18:51:00.940896 3435 heartbeater.cc:464] Registering TS with master...
I20260810 18:51:00.941465 3435 heartbeater.cc:511] Master 127.18.168.190:40999 requested a full tablet report, sending...
I20260810 18:51:00.942777 3060 ts_manager.cc:194] Registered new tserver with Master: 436302659835484193079dea06d4e462 (127.18.168.130:39573)
I20260810 18:51:00.943388 3060 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.130@KRBTEST.COM'} at 127.18.168.130:49413
W20260810 18:51:01.052621 3438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:51:01.052975 3438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:51:01.053051 3438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:51:01.057080 3438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:51:01.057191 3438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:51:01.057303 3438 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.131
I20260810 18:51:01.062880 3438 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:39893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.131:38147
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.168.131
--webserver_port=42871
--tserver_master_addrs=127.18.168.190:40999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.131
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:51:01.063961 3438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:51:01.065078 3438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:51:01.075202 3444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:01.075181 3443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:01.075279 3447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:01.075372 3438 server_base.cc:1034] running on GCE node
I20260810 18:51:01.075878 3438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:51:01.076546 3438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:51:01.077716 3438 hybrid_clock.cc:648] HybridClock initialized: now 1786387861077700 us; error 26 us; skew 500 ppm
Aug 10 18:51:01 dist-test-slave-cbsf krb5kdc[2388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387861, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:01.080497 3438 init.cc:377] Logged in from keytab as kudu/127.18.168.131@KRBTEST.COM (short username kudu)
I20260810 18:51:01.081440 3438 webserver.cc:460] Webserver started at http://127.18.168.131:42871/ using document root <none> and password file <none>
I20260810 18:51:01.081961 3438 fs_manager.cc:359] Metadata directory not provided
I20260810 18:51:01.082033 3438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:51:01.085872 3438 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260810 18:51:01.087877 3454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:01.089175 3438 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:51:01.089334 3438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/wal
uuid: "a6bbcf658b984e70bdc8f1826d4f8222"
format_stamp: "Formatted at 2026-08-10 18:51:00 on dist-test-slave-cbsf"
server_key: "a4250aa6aa5c15e39fbd8d699a003537"
server_key_iv: "264b2e4677ec8a84a2e3c4e964c90f3b"
server_key_version: "encryptionkey@0"
I20260810 18:51:01.089888 3438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:51:01.107957 3438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:51:01.110884 3438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:51:01.111145 3438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:51:01.111816 3438 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:51:01.112671 3438 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:51:01.112813 3438 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:01.112886 3438 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:51:01.112942 3438 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:01.122573 3438 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.131:38147
I20260810 18:51:01.122594 3592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.131:38147 every 8 connection(s)
I20260810 18:51:01.123610 3438 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
I20260810 18:51:01.131682 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 3438
Aug 10 18:51:01 dist-test-slave-cbsf krb5kdc[2388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387861, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:51:01.138681 3275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:01.125522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:34211 (local address 127.18.168.190:40999)
0810 18:51:01.125671 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:01.125675 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:01.126608 (+ 933us) server_negotiation.cc:407] Connection header received
0810 18:51:01.128001 (+ 1393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:01.128004 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:01.128061 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:01.128136 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:01.129451 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:01.129966 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:01.131082 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:01.131312 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:01.131393 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:01.134720 (+ 3327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:01.134722 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:01.134724 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:01.134746 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:01.136361 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:01.136987 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:01.136991 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:01.136993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:01.137082 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:01.137473 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:01.137476 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:01.137478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:01.137749 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:01.137872 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:01.138369 (+ 497us) server_negotiation.cc:299] Negotiation successful
0810 18:51:01.138508 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 18:51:01.139685 3596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:01.125912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.131:34211)
0810 18:51:01.126449 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:01.126491 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:01.127803 (+ 1312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:01.128347 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:01.128354 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:01.128833 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:51:01.129317 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:01.129331 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:01.130081 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:01.130083 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:01.130959 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:01.130968 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:01.131517 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:01.131522 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:01.131754 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:01.132751 (+ 997us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:51:01.132774 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:51:01.134548 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:51:01.136495 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:01.136501 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:01.136516 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:01.136875 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:01.137206 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:01.137209 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:01.137211 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:01.137359 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:01.137858 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:51:01.137865 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:51:01.138117 (+ 252us) client_negotiation.cc:769] Sending connection context
0810 18:51:01.138311 (+ 194us) client_negotiation.cc:240] Negotiation successful
0810 18:51:01.138645 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":158,"threads_started":1}
I20260810 18:51:01.140754 3598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:01.132411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59538 (local address 127.18.168.190:40999)
0810 18:51:01.132708 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:01.132713 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:01.132826 (+ 113us) server_negotiation.cc:407] Connection header received
0810 18:51:01.132913 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:01.132919 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:01.132969 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:01.133070 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:01.133889 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:01.134463 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:01.135402 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:01.135790 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:01.135910 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:01.137306 (+ 1396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:01.137310 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:01.137312 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:01.137343 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:01.138929 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:01.139426 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:01.139429 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:01.139430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:01.139504 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:01.139822 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:01.139827 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:01.139829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:01.140036 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:01.140249 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:01.140461 (+ 212us) server_negotiation.cc:299] Negotiation successful
0810 18:51:01.140583 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":148,"threads_started":1}
I20260810 18:51:01.141081 3594 heartbeater.cc:347] Connected to a master server at 127.18.168.190:40999
I20260810 18:51:01.141366 3594 heartbeater.cc:464] Registering TS with master...
I20260810 18:51:01.141963 3594 heartbeater.cc:511] Master 127.18.168.190:40999 requested a full tablet report, sending...
I20260810 18:51:01.143064 3060 ts_manager.cc:194] Registered new tserver with Master: a6bbcf658b984e70bdc8f1826d4f8222 (127.18.168.131:38147)
I20260810 18:51:01.143615 3060 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.131@KRBTEST.COM'} at 127.18.168.131:34211
I20260810 18:51:01.154361 19106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:51:01.158385 3598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:01.154997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59552 (local address 127.18.168.190:40999)
0810 18:51:01.155113 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:01.155117 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:01.155292 (+ 175us) server_negotiation.cc:407] Connection header received
0810 18:51:01.155415 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:01.155417 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:01.155507 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:01.155590 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:51:01.155791 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:01.156268 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:01.157177 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:01.157396 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:01.157476 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:01.157734 (+ 258us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:51:01.158082 (+ 348us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:51:01.158179 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:01.158234 (+ 55us) server_negotiation.cc:299] Negotiation successful
0810 18:51:01.158283 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 18:51:01.159030 3060 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:59552:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:51:01.162501 3060 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:51:01.170874 3194 tablet_service.cc:1467] Processing CreateTablet for tablet 1f63aa2e2ee44e1e830847a6d5e748ae (DEFAULT_TABLE table=test-table [id=d28e385e7a294fd382cf40dde9aff45c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:51:01.171663 3194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f63aa2e2ee44e1e830847a6d5e748ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:51:01.174777 3144 tablet_bootstrap.cc:492] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca: Bootstrap starting.
I20260810 18:51:01.175381 3603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:01.169913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:39573 (local address 127.0.0.1:47634)
0810 18:51:01.170175 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:01.170191 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:01.170277 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:01.171387 (+ 1110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:01.171390 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:01.171413 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:51:01.171516 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:01.171522 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:01.172987 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:01.172993 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:01.173838 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:01.173848 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:01.174951 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:01.174954 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:01.175043 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:01.175077 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 18:51:01.175133 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 18:51:01.175232 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":33,"thread_start_us":120,"threads_started":1}
I20260810 18:51:01.175722 3144 tablet_bootstrap.cc:654] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca: Neither blocks nor log segments found. Creating new log.
I20260810 18:51:01.176144 3605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:01.170373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47634 (local address 127.18.168.130:39573)
0810 18:51:01.170734 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:01.170745 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:01.170786 (+ 41us) server_negotiation.cc:407] Connection header received
0810 18:51:01.170864 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:01.170869 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:01.171101 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:01.171330 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:51:01.171648 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:01.172841 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:01.173976 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:01.174798 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:01.175682 (+ 884us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:01.175833 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:01.175886 (+ 53us) server_negotiation.cc:299] Negotiation successful
0810 18:51:01.175981 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":136,"threads_started":1}
I20260810 18:51:01.176227 3278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:01.169623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.131:38147 (local address 127.0.0.1:52682)
0810 18:51:01.169762 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:01.169780 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:01.169862 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:01.170964 (+ 1102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:01.170967 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:01.170986 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:51:01.171112 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:01.171120 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:01.173065 (+ 1945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:01.173069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:01.174189 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:01.174200 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:01.175867 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:01.175870 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:01.175955 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:01.175978 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 18:51:01.176028 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 18:51:01.176103 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":2}
I20260810 18:51:01.176862 3604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:01.169983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52682 (local address 127.18.168.131:38147)
0810 18:51:01.170424 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:01.170452 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:01.170480 (+ 28us) server_negotiation.cc:407] Connection header received
0810 18:51:01.170569 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:01.170575 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:01.170814 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:01.170993 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:51:01.171507 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:01.172885 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:01.174343 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:01.175725 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:01.176420 (+ 695us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:01.176551 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:01.176597 (+ 46us) server_negotiation.cc:299] Negotiation successful
0810 18:51:01.176728 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":187,"threads_started":1}
I20260810 18:51:01.178161 3144 tablet_bootstrap.cc:492] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca: No bootstrap required, opened a new log
I20260810 18:51:01.178252 3144 ts_tablet_manager.cc:1397] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260810 18:51:01.178982 3144 raft_consensus.cc:348] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6bbcf658b984e70bdc8f1826d4f8222" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 38147 } } peers { permanent_uuid: "436302659835484193079dea06d4e462" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39573 } } peers { permanent_uuid: "7c3c35e869e641a98116d03ab65f70ca" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37577 } }
I20260810 18:51:01.179114 3144 raft_consensus.cc:374] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:51:01.179163 3144 raft_consensus.cc:729] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c3c35e869e641a98116d03ab65f70ca, State: Initialized, Role: FOLLOWER
I20260810 18:51:01.179302 3144 consensus_queue.cc:260] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6bbcf658b984e70bdc8f1826d4f8222" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 38147 } } peers { permanent_uuid: "436302659835484193079dea06d4e462" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39573 } } peers { permanent_uuid: "7c3c35e869e641a98116d03ab65f70ca" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37577 } }
I20260810 18:51:01.179574 3144 ts_tablet_manager.cc:1428] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 18:51:01.179627 3357 tablet_service.cc:1467] Processing CreateTablet for tablet 1f63aa2e2ee44e1e830847a6d5e748ae (DEFAULT_TABLE table=test-table [id=d28e385e7a294fd382cf40dde9aff45c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:51:01.179804 3516 tablet_service.cc:1467] Processing CreateTablet for tablet 1f63aa2e2ee44e1e830847a6d5e748ae (DEFAULT_TABLE table=test-table [id=d28e385e7a294fd382cf40dde9aff45c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:51:01.180963 3357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f63aa2e2ee44e1e830847a6d5e748ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:51:01.181015 3516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f63aa2e2ee44e1e830847a6d5e748ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:51:01.188004 3608 tablet_bootstrap.cc:492] T 1f63aa2e2ee44e1e830847a6d5e748ae P a6bbcf658b984e70bdc8f1826d4f8222: Bootstrap starting.
I20260810 18:51:01.188035 3609 tablet_bootstrap.cc:492] T 1f63aa2e2ee44e1e830847a6d5e748ae P 436302659835484193079dea06d4e462: Bootstrap starting.
I20260810 18:51:01.190511 3608 tablet_bootstrap.cc:654] T 1f63aa2e2ee44e1e830847a6d5e748ae P a6bbcf658b984e70bdc8f1826d4f8222: Neither blocks nor log segments found. Creating new log.
I20260810 18:51:01.190649 3609 tablet_bootstrap.cc:654] T 1f63aa2e2ee44e1e830847a6d5e748ae P 436302659835484193079dea06d4e462: Neither blocks nor log segments found. Creating new log.
I20260810 18:51:01.191398 3608 log.cc:824] T 1f63aa2e2ee44e1e830847a6d5e748ae P a6bbcf658b984e70bdc8f1826d4f8222: Log is configured to *not* fsync() on all Append() calls
I20260810 18:51:01.191418 3609 log.cc:824] T 1f63aa2e2ee44e1e830847a6d5e748ae P 436302659835484193079dea06d4e462: Log is configured to *not* fsync() on all Append() calls
I20260810 18:51:01.193490 3609 tablet_bootstrap.cc:492] T 1f63aa2e2ee44e1e830847a6d5e748ae P 436302659835484193079dea06d4e462: No bootstrap required, opened a new log
I20260810 18:51:01.193545 3608 tablet_bootstrap.cc:492] T 1f63aa2e2ee44e1e830847a6d5e748ae P a6bbcf658b984e70bdc8f1826d4f8222: No bootstrap required, opened a new log
I20260810 18:51:01.193732 3609 ts_tablet_manager.cc:1397] T 1f63aa2e2ee44e1e830847a6d5e748ae P 436302659835484193079dea06d4e462: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 18:51:01.193754 3608 ts_tablet_manager.cc:1397] T 1f63aa2e2ee44e1e830847a6d5e748ae P a6bbcf658b984e70bdc8f1826d4f8222: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260810 18:51:01.196574 3609 raft_consensus.cc:348] T 1f63aa2e2ee44e1e830847a6d5e748ae P 436302659835484193079dea06d4e462 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6bbcf658b984e70bdc8f1826d4f8222" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 38147 } } peers { permanent_uuid: "436302659835484193079dea06d4e462" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39573 } } peers { permanent_uuid: "7c3c35e869e641a98116d03ab65f70ca" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37577 } }
I20260810 18:51:01.196573 3608 raft_consensus.cc:348] T 1f63aa2e2ee44e1e830847a6d5e748ae P a6bbcf658b984e70bdc8f1826d4f8222 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6bbcf658b984e70bdc8f1826d4f8222" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 38147 } } peers { permanent_uuid: "436302659835484193079dea06d4e462" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39573 } } peers { permanent_uuid: "7c3c35e869e641a98116d03ab65f70ca" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37577 } }
I20260810 18:51:01.196851 3609 raft_consensus.cc:374] T 1f63aa2e2ee44e1e830847a6d5e748ae P 436302659835484193079dea06d4e462 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:51:01.196872 3608 raft_consensus.cc:374] T 1f63aa2e2ee44e1e830847a6d5e748ae P a6bbcf658b984e70bdc8f1826d4f8222 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:51:01.196930 3608 raft_consensus.cc:729] T 1f63aa2e2ee44e1e830847a6d5e748ae P a6bbcf658b984e70bdc8f1826d4f8222 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6bbcf658b984e70bdc8f1826d4f8222, State: Initialized, Role: FOLLOWER
I20260810 18:51:01.196928 3609 raft_consensus.cc:729] T 1f63aa2e2ee44e1e830847a6d5e748ae P 436302659835484193079dea06d4e462 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 436302659835484193079dea06d4e462, State: Initialized, Role: FOLLOWER
I20260810 18:51:01.197384 3608 consensus_queue.cc:260] T 1f63aa2e2ee44e1e830847a6d5e748ae P a6bbcf658b984e70bdc8f1826d4f8222 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6bbcf658b984e70bdc8f1826d4f8222" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 38147 } } peers { permanent_uuid: "436302659835484193079dea06d4e462" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39573 } } peers { permanent_uuid: "7c3c35e869e641a98116d03ab65f70ca" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37577 } }
I20260810 18:51:01.197384 3609 consensus_queue.cc:260] T 1f63aa2e2ee44e1e830847a6d5e748ae P 436302659835484193079dea06d4e462 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6bbcf658b984e70bdc8f1826d4f8222" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 38147 } } peers { permanent_uuid: "436302659835484193079dea06d4e462" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39573 } } peers { permanent_uuid: "7c3c35e869e641a98116d03ab65f70ca" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37577 } }
I20260810 18:51:01.198073 3435 heartbeater.cc:503] Master 127.18.168.190:40999 was elected leader, sending a full tablet report...
I20260810 18:51:01.198073 3594 heartbeater.cc:503] Master 127.18.168.190:40999 was elected leader, sending a full tablet report...
I20260810 18:51:01.198395 3608 ts_tablet_manager.cc:1428] T 1f63aa2e2ee44e1e830847a6d5e748ae P a6bbcf658b984e70bdc8f1826d4f8222: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260810 18:51:01.199028 3609 ts_tablet_manager.cc:1428] T 1f63aa2e2ee44e1e830847a6d5e748ae P 436302659835484193079dea06d4e462: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260810 18:51:01.219208 3273 tablet.cc:2369] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:51:01.357980 3613 raft_consensus.cc:482] T 1f63aa2e2ee44e1e830847a6d5e748ae P 436302659835484193079dea06d4e462 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:51:01.358187 3613 raft_consensus.cc:504] T 1f63aa2e2ee44e1e830847a6d5e748ae P 436302659835484193079dea06d4e462 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6bbcf658b984e70bdc8f1826d4f8222" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 38147 } } peers { permanent_uuid: "436302659835484193079dea06d4e462" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39573 } } peers { permanent_uuid: "7c3c35e869e641a98116d03ab65f70ca" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37577 } }
I20260810 18:51:01.359340 3613 leader_election.cc:288] T 1f63aa2e2ee44e1e830847a6d5e748ae P 436302659835484193079dea06d4e462 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6bbcf658b984e70bdc8f1826d4f8222 (127.18.168.131:38147), 7c3c35e869e641a98116d03ab65f70ca (127.18.168.129:37577)
I20260810 18:51:01.363262 3604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:01.359807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:56483 (local address 127.18.168.131:38147)
0810 18:51:01.359988 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:01.359994 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:01.360014 (+ 20us) server_negotiation.cc:407] Connection header received
0810 18:51:01.360100 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:01.360105 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:01.360184 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:01.360273 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:51:01.360587 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:01.361182 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:01.362249 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:01.362663 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:01.362799 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:01.362851 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:01.363062 (+ 211us) server_negotiation.cc:299] Negotiation successful
0810 18:51:01.363132 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260810 18:51:01.363260 3437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:01.359649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.131:38147 (local address 127.18.168.130:56483)
0810 18:51:01.359810 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:01.359831 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:01.359935 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:01.360296 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:01.360299 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:01.360317 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:51:01.360457 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:01.360464 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:01.361315 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:01.361320 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:01.362109 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:01.362118 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:01.362793 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:01.362796 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:01.362903 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:01.362932 (+ 29us) client_negotiation.cc:769] Sending connection context
0810 18:51:01.363022 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 18:51:01.363095 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260810 18:51:01.363696 3614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:01.359978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:37577 (local address 127.18.168.130:42939)
0810 18:51:01.360365 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:01.360380 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:01.360472 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:01.360816 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:01.360819 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:01.360836 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:51:01.360961 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:01.360968 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:01.361737 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:01.361741 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:01.362524 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:01.362533 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:01.363262 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:01.363266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:01.363373 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:01.363396 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 18:51:01.363468 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 18:51:01.363532 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":133,"threads_started":1}
I20260810 18:51:01.363818 3615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:01.360108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:42939 (local address 127.18.168.129:37577)
0810 18:51:01.360454 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:01.360458 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:01.360472 (+ 14us) server_negotiation.cc:407] Connection header received
0810 18:51:01.360595 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:01.360598 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:01.360653 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:01.360803 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:51:01.361093 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:01.361605 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:01.362674 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:01.363085 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:01.363269 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:01.363317 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:01.363573 (+ 256us) server_negotiation.cc:299] Negotiation successful
0810 18:51:01.363689 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":148,"threads_started":1}
I20260810 18:51:01.363961 3538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f63aa2e2ee44e1e830847a6d5e748ae" candidate_uuid: "436302659835484193079dea06d4e462" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6bbcf658b984e70bdc8f1826d4f8222" is_pre_election: true
I20260810 18:51:01.364308 3538 raft_consensus.cc:2454] T 1f63aa2e2ee44e1e830847a6d5e748ae P a6bbcf658b984e70bdc8f1826d4f8222 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 436302659835484193079dea06d4e462 in term 0.
I20260810 18:51:01.364372 3219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f63aa2e2ee44e1e830847a6d5e748ae" candidate_uuid: "436302659835484193079dea06d4e462" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c3c35e869e641a98116d03ab65f70ca" is_pre_election: true
I20260810 18:51:01.364753 3219 raft_consensus.cc:2454] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 436302659835484193079dea06d4e462 in term 0.
I20260810 18:51:01.364871 3301 leader_election.cc:302] T 1f63aa2e2ee44e1e830847a6d5e748ae P 436302659835484193079dea06d4e462 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 436302659835484193079dea06d4e462, a6bbcf658b984e70bdc8f1826d4f8222; no voters:
I20260810 18:51:01.365191 3613 raft_consensus.cc:2781] T 1f63aa2e2ee44e1e830847a6d5e748ae P 436302659835484193079dea06d4e462 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:51:01.365284 3613 raft_consensus.cc:482] T 1f63aa2e2ee44e1e830847a6d5e748ae P 436302659835484193079dea06d4e462 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:51:01.365334 3613 raft_consensus.cc:3037] T 1f63aa2e2ee44e1e830847a6d5e748ae P 436302659835484193079dea06d4e462 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:51:01.366384 3613 raft_consensus.cc:504] T 1f63aa2e2ee44e1e830847a6d5e748ae P 436302659835484193079dea06d4e462 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6bbcf658b984e70bdc8f1826d4f8222" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 38147 } } peers { permanent_uuid: "436302659835484193079dea06d4e462" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39573 } } peers { permanent_uuid: "7c3c35e869e641a98116d03ab65f70ca" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37577 } }
I20260810 18:51:01.366747 3613 leader_election.cc:288] T 1f63aa2e2ee44e1e830847a6d5e748ae P 436302659835484193079dea06d4e462 [CANDIDATE]: Term 1 election: Requested vote from peers a6bbcf658b984e70bdc8f1826d4f8222 (127.18.168.131:38147), 7c3c35e869e641a98116d03ab65f70ca (127.18.168.129:37577)
I20260810 18:51:01.367086 3538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f63aa2e2ee44e1e830847a6d5e748ae" candidate_uuid: "436302659835484193079dea06d4e462" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6bbcf658b984e70bdc8f1826d4f8222"
I20260810 18:51:01.367129 3219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f63aa2e2ee44e1e830847a6d5e748ae" candidate_uuid: "436302659835484193079dea06d4e462" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c3c35e869e641a98116d03ab65f70ca"
I20260810 18:51:01.367233 3538 raft_consensus.cc:3037] T 1f63aa2e2ee44e1e830847a6d5e748ae P a6bbcf658b984e70bdc8f1826d4f8222 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:51:01.367235 3219 raft_consensus.cc:3037] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:51:01.368275 3219 raft_consensus.cc:2454] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 436302659835484193079dea06d4e462 in term 1.
I20260810 18:51:01.368366 3538 raft_consensus.cc:2454] T 1f63aa2e2ee44e1e830847a6d5e748ae P a6bbcf658b984e70bdc8f1826d4f8222 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 436302659835484193079dea06d4e462 in term 1.
I20260810 18:51:01.368553 3301 leader_election.cc:302] T 1f63aa2e2ee44e1e830847a6d5e748ae P 436302659835484193079dea06d4e462 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 436302659835484193079dea06d4e462, 7c3c35e869e641a98116d03ab65f70ca; no voters:
I20260810 18:51:01.368759 3613 raft_consensus.cc:2781] T 1f63aa2e2ee44e1e830847a6d5e748ae P 436302659835484193079dea06d4e462 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:51:01.368981 3613 raft_consensus.cc:686] T 1f63aa2e2ee44e1e830847a6d5e748ae P 436302659835484193079dea06d4e462 [term 1 LEADER]: Becoming Leader. State: Replica: 436302659835484193079dea06d4e462, State: Running, Role: LEADER
I20260810 18:51:01.369325 3613 consensus_queue.cc:237] T 1f63aa2e2ee44e1e830847a6d5e748ae P 436302659835484193079dea06d4e462 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6bbcf658b984e70bdc8f1826d4f8222" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 38147 } } peers { permanent_uuid: "436302659835484193079dea06d4e462" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39573 } } peers { permanent_uuid: "7c3c35e869e641a98116d03ab65f70ca" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37577 } }
I20260810 18:51:01.372741 3060 catalog_manager.cc:5495] T 1f63aa2e2ee44e1e830847a6d5e748ae P 436302659835484193079dea06d4e462 reported cstate change: term changed from 0 to 1, leader changed from <none> to 436302659835484193079dea06d4e462 (127.18.168.130). New cstate: current_term: 1 leader_uuid: "436302659835484193079dea06d4e462" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6bbcf658b984e70bdc8f1826d4f8222" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 38147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "436302659835484193079dea06d4e462" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c3c35e869e641a98116d03ab65f70ca" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37577 } health_report { overall_health: UNKNOWN } } }
W20260810 18:51:01.374569 3595 tablet.cc:2369] T 1f63aa2e2ee44e1e830847a6d5e748ae P a6bbcf658b984e70bdc8f1826d4f8222: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:51:01.410931 3605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:01.407793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47636 (local address 127.18.168.130:39573)
0810 18:51:01.407942 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:01.407946 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:01.407961 (+ 15us) server_negotiation.cc:407] Connection header received
0810 18:51:01.408035 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:01.408037 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:01.408090 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:01.408186 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:51:01.408391 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:01.408899 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:01.409677 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:01.409875 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:01.409948 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:01.410233 (+ 285us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:51:01.410617 (+ 384us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:51:01.410701 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:01.410764 (+ 63us) server_negotiation.cc:299] Negotiation successful
0810 18:51:01.410812 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 18:51:01.416750 3538 raft_consensus.cc:1261] T 1f63aa2e2ee44e1e830847a6d5e748ae P a6bbcf658b984e70bdc8f1826d4f8222 [term 1 FOLLOWER]: Refusing update from remote peer 436302659835484193079dea06d4e462: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:51:01.416788 3219 raft_consensus.cc:1261] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [term 1 FOLLOWER]: Refusing update from remote peer 436302659835484193079dea06d4e462: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:51:01.417508 3613 consensus_queue.cc:1035] T 1f63aa2e2ee44e1e830847a6d5e748ae P 436302659835484193079dea06d4e462 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c3c35e869e641a98116d03ab65f70ca" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:51:01.417706 3616 consensus_queue.cc:1035] T 1f63aa2e2ee44e1e830847a6d5e748ae P 436302659835484193079dea06d4e462 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6bbcf658b984e70bdc8f1826d4f8222" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 38147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:51:01.424345 3620 mvcc.cc:205] Tried to move back new op lower bound from 7317044680355524608 to 7317044680171868160. Current Snapshot: MvccSnapshot[applied={T|T < 7317044680355524608}]
I20260810 18:51:01.426275 3621 mvcc.cc:205] Tried to move back new op lower bound from 7317044680355524608 to 7317044680171868160. Current Snapshot: MvccSnapshot[applied={T|T < 7317044680355524608}]
I20260810 18:51:01.433053 3057 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:59552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:51:01.433187 3057 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:59552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:51:01.435520 3057 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2: Sending DeleteTablet for 3 replicas of tablet 1f63aa2e2ee44e1e830847a6d5e748ae
I20260810 18:51:01.436257 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 3099
I20260810 18:51:01.436211 3357 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f63aa2e2ee44e1e830847a6d5e748ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:51:01 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:47634
I20260810 18:51:01.436384 3516 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f63aa2e2ee44e1e830847a6d5e748ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:51:01 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:52682
I20260810 18:51:01.442162 3632 tablet_replica.cc:330] T 1f63aa2e2ee44e1e830847a6d5e748ae P a6bbcf658b984e70bdc8f1826d4f8222: stopping tablet replica
I20260810 18:51:01.442528 3632 raft_consensus.cc:2229] T 1f63aa2e2ee44e1e830847a6d5e748ae P a6bbcf658b984e70bdc8f1826d4f8222 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:51:01.442929 3632 raft_consensus.cc:2258] T 1f63aa2e2ee44e1e830847a6d5e748ae P a6bbcf658b984e70bdc8f1826d4f8222 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:51:01.443315 3630 tablet_replica.cc:330] T 1f63aa2e2ee44e1e830847a6d5e748ae P 436302659835484193079dea06d4e462: stopping tablet replica
I20260810 18:51:01.443567 3630 raft_consensus.cc:2229] T 1f63aa2e2ee44e1e830847a6d5e748ae P 436302659835484193079dea06d4e462 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:51:01.443957 3630 raft_consensus.cc:2258] T 1f63aa2e2ee44e1e830847a6d5e748ae P 436302659835484193079dea06d4e462 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:51:01.444761 3632 ts_tablet_manager.cc:1905] T 1f63aa2e2ee44e1e830847a6d5e748ae P a6bbcf658b984e70bdc8f1826d4f8222: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:51:01.445766 3630 ts_tablet_manager.cc:1905] T 1f63aa2e2ee44e1e830847a6d5e748ae P 436302659835484193079dea06d4e462: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:51:01.448364 3632 ts_tablet_manager.cc:1918] T 1f63aa2e2ee44e1e830847a6d5e748ae P a6bbcf658b984e70bdc8f1826d4f8222: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:51:01.448428 3630 ts_tablet_manager.cc:1918] T 1f63aa2e2ee44e1e830847a6d5e748ae P 436302659835484193079dea06d4e462: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:51:01.448648 3630 log.cc:1196] T 1f63aa2e2ee44e1e830847a6d5e748ae P 436302659835484193079dea06d4e462: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/wal/wals/1f63aa2e2ee44e1e830847a6d5e748ae
I20260810 18:51:01.448870 3632 log.cc:1196] T 1f63aa2e2ee44e1e830847a6d5e748ae P a6bbcf658b984e70bdc8f1826d4f8222: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/wal/wals/1f63aa2e2ee44e1e830847a6d5e748ae
I20260810 18:51:01.449219 3630 ts_tablet_manager.cc:1939] T 1f63aa2e2ee44e1e830847a6d5e748ae P 436302659835484193079dea06d4e462: Deleting consensus metadata
I20260810 18:51:01.449283 3632 ts_tablet_manager.cc:1939] T 1f63aa2e2ee44e1e830847a6d5e748ae P a6bbcf658b984e70bdc8f1826d4f8222: Deleting consensus metadata
W20260810 18:51:01.449956 3036 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.168.129:37577 (error 108)
I20260810 18:51:01.450546 3038 catalog_manager.cc:4841] TS 436302659835484193079dea06d4e462 (127.18.168.130:39573): tablet 1f63aa2e2ee44e1e830847a6d5e748ae (table test-table [id=d28e385e7a294fd382cf40dde9aff45c]) successfully deleted
I20260810 18:51:01.450579 3036 catalog_manager.cc:4841] TS a6bbcf658b984e70bdc8f1826d4f8222 (127.18.168.131:38147): tablet 1f63aa2e2ee44e1e830847a6d5e748ae (table test-table [id=d28e385e7a294fd382cf40dde9aff45c]) successfully deleted
I20260810 18:51:01.451203 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 3276
I20260810 18:51:01.451570 3278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:01.451237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:37577 (local address 127.0.0.1:34690)
0810 18:51:01.451419 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:01.451489 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.129:37577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
W20260810 18:51:01.453624 3036 catalog_manager.cc:4584] TS 7c3c35e869e641a98116d03ab65f70ca (127.18.168.129:37577): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1f63aa2e2ee44e1e830847a6d5e748ae: Network error: Client connection negotiation failed: client connection to 127.18.168.129:37577: connect: Connection refused (error 111)
I20260810 18:51:01.459993 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 3438
I20260810 18:51:01.468214 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 3017
I20260810 18:51:01.477093 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:40999
--webserver_interface=127.18.168.190
--webserver_port=40807
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:39893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:40999
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:51:01.593384 3634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:51:01.593608 3634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:51:01.593647 3634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:51:01.597592 3634 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:51:01.597637 3634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:51:01.597656 3634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:51:01.597672 3634 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:51:01.597741 3634 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:51:01.602875 3634 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:39893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:40999
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:40999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=40807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:51:01.603811 3634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:51:01.604799 3634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:51:01.613157 3640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:01.613173 3639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:01.613209 3642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:01.613330 3634 server_base.cc:1034] running on GCE node
I20260810 18:51:01.614154 3634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:51:01.615180 3634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:51:01.616375 3634 hybrid_clock.cc:648] HybridClock initialized: now 1786387861616351 us; error 46 us; skew 500 ppm
Aug 10 18:51:01 dist-test-slave-cbsf krb5kdc[2388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387861, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:01.619247 3634 init.cc:377] Logged in from keytab as kudu/127.18.168.190@KRBTEST.COM (short username kudu)
I20260810 18:51:01.620100 3634 webserver.cc:460] Webserver started at http://127.18.168.190:40807/ using document root <none> and password file <none>
I20260810 18:51:01.620640 3634 fs_manager.cc:359] Metadata directory not provided
I20260810 18:51:01.620771 3634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:51:01.624410 3634 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.002s
I20260810 18:51:01.626549 3649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:01.627774 3634 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 18:51:01.627928 3634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "69089a0bf06544d4bbca5fc069539ac2"
format_stamp: "Formatted at 2026-08-10 18:50:59 on dist-test-slave-cbsf"
server_key: "9d15b64b5cca04f3ec706f523022d8e1"
server_key_iv: "da562ea5a3130604866f91c09367e839"
server_key_version: "encryptionkey@0"
I20260810 18:51:01.628484 3634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:51:01.656908 3634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:51:01.660322 3634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:51:01.660557 3634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:51:01.667826 3634 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:40999
I20260810 18:51:01.668144 3711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:40999 every 8 connection(s)
I20260810 18:51:01.669025 3634 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:51:01.673249 3712 sys_catalog.cc:263] Verifying existing consensus state
I20260810 18:51:01.674598 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 3634
I20260810 18:51:01.675097 3712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2: Bootstrap starting.
I20260810 18:51:01.675096 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.129:37577
--local_ip_for_outbound_sockets=127.18.168.129
--tserver_master_addrs=127.18.168.190:40999
--webserver_port=41297
--webserver_interface=127.18.168.129
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:39893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:51:01.694165 3712 log.cc:824] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2: Log is configured to *not* fsync() on all Append() calls
I20260810 18:51:01.705761 3712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:51:01.706173 3712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2: Bootstrap complete.
I20260810 18:51:01.710398 3712 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69089a0bf06544d4bbca5fc069539ac2" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 40999 } }
I20260810 18:51:01.710769 3712 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69089a0bf06544d4bbca5fc069539ac2, State: Initialized, Role: FOLLOWER
I20260810 18:51:01.711247 3712 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69089a0bf06544d4bbca5fc069539ac2" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 40999 } }
I20260810 18:51:01.711357 3712 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:51:01.711447 3712 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:51:01.711555 3712 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [term 2 FOLLOWER]: Advancing to term 3
I20260810 18:51:01.713446 3712 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69089a0bf06544d4bbca5fc069539ac2" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 40999 } }
I20260810 18:51:01.713804 3712 leader_election.cc:302] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69089a0bf06544d4bbca5fc069539ac2; no voters:
I20260810 18:51:01.714144 3712 leader_election.cc:288] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 18:51:01.714226 3718 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 18:51:01.714435 3718 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [term 3 LEADER]: Becoming Leader. State: Replica: 69089a0bf06544d4bbca5fc069539ac2, State: Running, Role: LEADER
I20260810 18:51:01.714720 3718 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69089a0bf06544d4bbca5fc069539ac2" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 40999 } }
I20260810 18:51:01.715356 3712 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:51:01.715667 3718 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69089a0bf06544d4bbca5fc069539ac2. Latest consensus state: current_term: 3 leader_uuid: "69089a0bf06544d4bbca5fc069539ac2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69089a0bf06544d4bbca5fc069539ac2" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 40999 } } }
I20260810 18:51:01.715797 3718 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [sys.catalog]: This master's current role is: LEADER
I20260810 18:51:01.716199 3724 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:51:01.716195 3720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "69089a0bf06544d4bbca5fc069539ac2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69089a0bf06544d4bbca5fc069539ac2" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 40999 } } }
I20260810 18:51:01.716475 3720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69089a0bf06544d4bbca5fc069539ac2 [sys.catalog]: This master's current role is: LEADER
I20260810 18:51:01.721458 3724 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:51:01.722122 3724 catalog_manager.cc:1259] Loaded cluster ID: 9dc514d0f3ce4c8da0755be8b4a96313
I20260810 18:51:01.722185 3724 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260810 18:51:01.723301 3724 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f1649a0d9ad google::LogMessage::Fail() at ??:0
@ 0x7f1649a1088a google::LogMessage::SendToLog() at ??:0
@ 0x7f1649a0d4de google::LogMessage::Flush() at ??:0
@ 0x7f1649a0ea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f165434cfdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f165434dd2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f165434a579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f165438e640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f164bc5f3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f164bc5fc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f164bc613f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f164bc50512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f164d2946db start_thread at ??:0
@ 0x7f164850b61f clone at ??:0
W20260810 18:51:01.810516 3716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:51:01.810799 3716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:51:01.810882 3716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:51:01.815055 3716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:51:01.815141 3716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:51:01.815236 3716 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.129
I20260810 18:51:01.820622 3716 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:39893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.129:37577
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.168.129
--webserver_port=41297
--tserver_master_addrs=127.18.168.190:40999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.129
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:51:01.821836 3716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:51:01.822898 3716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:51:01.833381 3767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:01.833389 3766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:01.833429 3769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:01.833863 3716 server_base.cc:1034] running on GCE node
I20260810 18:51:01.834309 3716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:51:01.835041 3716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:51:01.836248 3716 hybrid_clock.cc:648] HybridClock initialized: now 1786387861836218 us; error 43 us; skew 500 ppm
Aug 10 18:51:01 dist-test-slave-cbsf krb5kdc[2388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387861, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:01.839377 3716 init.cc:377] Logged in from keytab as kudu/127.18.168.129@KRBTEST.COM (short username kudu)
I20260810 18:51:01.840348 3716 webserver.cc:460] Webserver started at http://127.18.168.129:41297/ using document root <none> and password file <none>
I20260810 18:51:01.841048 3716 fs_manager.cc:359] Metadata directory not provided
I20260810 18:51:01.841158 3716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:51:01.845316 3716 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260810 18:51:01.847668 3776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:01.849113 3716 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:51:01.849284 3716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/wal
uuid: "7c3c35e869e641a98116d03ab65f70ca"
format_stamp: "Formatted at 2026-08-10 18:50:59 on dist-test-slave-cbsf"
server_key: "2cf66e56f2f034576bd6dd15490fd60e"
server_key_iv: "c32eb7f3ba0919eb3ffb84505d19d9fe"
server_key_version: "encryptionkey@0"
I20260810 18:51:01.849859 3716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:51:01.863958 3716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:51:01.867543 3716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:51:01.867782 3716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:51:01.868455 3716 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:51:01.869735 3784 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 18:51:01.875590 3716 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 18:51:01.875692 3716 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260810 18:51:01.875778 3716 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 18:51:01.877619 3716 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 18:51:01.877698 3716 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260810 18:51:01.878000 3784 tablet_bootstrap.cc:492] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca: Bootstrap starting.
I20260810 18:51:01.889066 3716 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.129:37577
I20260810 18:51:01.889379 3913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.129:37577 every 8 connection(s)
I20260810 18:51:01.890098 3716 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
I20260810 18:51:01.893581 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 3716
I20260810 18:51:01.894104 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.130:39573
--local_ip_for_outbound_sockets=127.18.168.130
--tserver_master_addrs=127.18.168.190:40999
--webserver_port=39067
--webserver_interface=127.18.168.130
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:39893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:51:01.901849 3784 log.cc:824] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca: Log is configured to *not* fsync() on all Append() calls
I20260810 18:51:01.911326 3784 tablet_bootstrap.cc:492] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:51:01.911758 3784 tablet_bootstrap.cc:492] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca: Bootstrap complete.
I20260810 18:51:01.913439 3784 ts_tablet_manager.cc:1397] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca: Time spent bootstrapping tablet: real 0.036s user 0.026s sys 0.008s
I20260810 18:51:01.916872 3784 raft_consensus.cc:348] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6bbcf658b984e70bdc8f1826d4f8222" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 38147 } } peers { permanent_uuid: "436302659835484193079dea06d4e462" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39573 } } peers { permanent_uuid: "7c3c35e869e641a98116d03ab65f70ca" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37577 } }
I20260810 18:51:01.917491 3784 raft_consensus.cc:729] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c3c35e869e641a98116d03ab65f70ca, State: Initialized, Role: FOLLOWER
I20260810 18:51:01.918047 3784 consensus_queue.cc:260] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6bbcf658b984e70bdc8f1826d4f8222" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 38147 } } peers { permanent_uuid: "436302659835484193079dea06d4e462" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39573 } } peers { permanent_uuid: "7c3c35e869e641a98116d03ab65f70ca" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37577 } }
I20260810 18:51:01.919142 3784 ts_tablet_manager.cc:1428] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260810 18:51:01.956780 3917 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:51:01.892063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.129:46929)
0810 18:51:01.892581 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:01.892635 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:01.893640 (+ 1005us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:01.955967 (+ 62327us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":170,"threads_started":1}
W20260810 18:51:01.957309 3914 heartbeater.cc:644] Failed to heartbeat to 127.18.168.190:40999 (0 consecutive failures): Network error: Failed to ping master at 127.18.168.190:40999: Client connection negotiation failed: client connection to 127.18.168.190:40999: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 18:51:01.958097 3917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:01.957847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.129:51547)
0810 18:51:01.957982 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:01.958046 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260810 18:51:01.958802 3917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:01.958589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.129:54983)
0810 18:51:01.958705 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:01.958756 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 18:51:01.958981 3914 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 18:51:02.016050 3918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:51:02.016367 3918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:51:02.016446 3918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:51:02.020488 3918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:51:02.020591 3918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:51:02.020707 3918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.130
I20260810 18:51:02.026108 3918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:39893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.130:39573
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.168.130
--webserver_port=39067
--tserver_master_addrs=127.18.168.190:40999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.130
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:51:02.027182 3918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:51:02.028366 3918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:51:02.038537 3926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:02.038524 3928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:02.038563 3925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:02.038903 3918 server_base.cc:1034] running on GCE node
I20260810 18:51:02.039318 3918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:51:02.039976 3918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:51:02.041211 3918 hybrid_clock.cc:648] HybridClock initialized: now 1786387862041182 us; error 30 us; skew 500 ppm
Aug 10 18:51:02 dist-test-slave-cbsf krb5kdc[2388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387862, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:02.043917 3918 init.cc:377] Logged in from keytab as kudu/127.18.168.130@KRBTEST.COM (short username kudu)
I20260810 18:51:02.044843 3918 webserver.cc:460] Webserver started at http://127.18.168.130:39067/ using document root <none> and password file <none>
I20260810 18:51:02.045409 3918 fs_manager.cc:359] Metadata directory not provided
I20260810 18:51:02.045507 3918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:51:02.049322 3918 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260810 18:51:02.051426 3937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:02.052641 3918 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:51:02.052806 3918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/wal
uuid: "436302659835484193079dea06d4e462"
format_stamp: "Formatted at 2026-08-10 18:50:59 on dist-test-slave-cbsf"
server_key: "102fe108fde9dcf0179f2fec6854b3a3"
server_key_iv: "9b39e195684ab7b07072f6f3d835150d"
server_key_version: "encryptionkey@0"
I20260810 18:51:02.053450 3918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:51:02.073107 3918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:51:02.076056 3918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:51:02.076269 3918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:51:02.076967 3918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:51:02.077836 3918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:51:02.077913 3918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:02.077978 3918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:51:02.078011 3918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:02.087471 3918 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.130:39573
I20260810 18:51:02.087507 4068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.130:39573 every 8 connection(s)
I20260810 18:51:02.088470 3918 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
I20260810 18:51:02.092337 4073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:02.090531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.130:58125)
0810 18:51:02.090991 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:02.091114 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":129,"threads_started":1}
W20260810 18:51:02.092837 4069 heartbeater.cc:644] Failed to heartbeat to 127.18.168.190:40999 (0 consecutive failures): Network error: Failed to ping master at 127.18.168.190:40999: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
I20260810 18:51:02.093441 4073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:02.093221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.130:40247)
0810 18:51:02.093339 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:02.093393 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260810 18:51:02.094111 4073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:02.093893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.130:49453)
0810 18:51:02.094003 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:02.094061 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 18:51:02.094311 4069 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 18:51:02.094873 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 3918
I20260810 18:51:02.095292 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.131:38147
--local_ip_for_outbound_sockets=127.18.168.131
--tserver_master_addrs=127.18.168.190:40999
--webserver_port=42871
--webserver_interface=127.18.168.131
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:39893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:51:02.213618 4075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:51:02.213871 4075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:51:02.213912 4075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:51:02.217882 4075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:51:02.217938 4075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:51:02.217996 4075 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.131
I20260810 18:51:02.223114 4075 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:39893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.131:38147
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.168.131
--webserver_port=42871
--tserver_master_addrs=127.18.168.190:40999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.131
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:51:02.224125 4075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:51:02.225185 4075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:51:02.235428 4081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:02.235543 4080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:02.235630 4083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:02.235778 4075 server_base.cc:1034] running on GCE node
I20260810 18:51:02.236153 4075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:51:02.236860 4075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:51:02.238022 4075 hybrid_clock.cc:648] HybridClock initialized: now 1786387862238009 us; error 24 us; skew 500 ppm
Aug 10 18:51:02 dist-test-slave-cbsf krb5kdc[2388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387862, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:02.240876 4075 init.cc:377] Logged in from keytab as kudu/127.18.168.131@KRBTEST.COM (short username kudu)
I20260810 18:51:02.241762 4075 webserver.cc:460] Webserver started at http://127.18.168.131:42871/ using document root <none> and password file <none>
I20260810 18:51:02.242342 4075 fs_manager.cc:359] Metadata directory not provided
I20260810 18:51:02.242403 4075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:51:02.246349 4075 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:51:02.248436 4091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:02.249706 4075 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:51:02.249856 4075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/wal
uuid: "a6bbcf658b984e70bdc8f1826d4f8222"
format_stamp: "Formatted at 2026-08-10 18:51:00 on dist-test-slave-cbsf"
server_key: "a4250aa6aa5c15e39fbd8d699a003537"
server_key_iv: "264b2e4677ec8a84a2e3c4e964c90f3b"
server_key_version: "encryptionkey@0"
I20260810 18:51:02.250411 4075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:51:02.267891 4075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:51:02.271344 4075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:51:02.271591 4075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:51:02.272212 4075 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:51:02.273144 4075 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:51:02.273233 4075 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:02.273315 4075 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:51:02.273353 4075 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:02.282809 4075 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.131:38147
I20260810 18:51:02.282845 4221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.131:38147 every 8 connection(s)
I20260810 18:51:02.283806 4075 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
I20260810 18:51:02.287649 4225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:02.285904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.131:38393)
0810 18:51:02.286386 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:02.286502 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":137,"threads_started":1}
W20260810 18:51:02.288158 4222 heartbeater.cc:644] Failed to heartbeat to 127.18.168.190:40999 (0 consecutive failures): Network error: Failed to ping master at 127.18.168.190:40999: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
I20260810 18:51:02.288807 4225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:02.288559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.131:36353)
0810 18:51:02.288675 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:02.288756 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 18:51:02.289429 4225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:02.289220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.131:34891)
0810 18:51:02.289331 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:02.289382 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260810 18:51:02.289626 4222 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 18:51:02.293128 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 4075
I20260810 18:51:02.960070 4227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:02.959662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.129:56857)
0810 18:51:02.959921 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:02.959971 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":83,"threads_started":1}
I20260810 18:51:03.095474 4228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:03.094920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.130:55605)
0810 18:51:03.095225 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:03.095318 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":110,"threads_started":1}
I20260810 18:51:03.290571 4229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:03.290134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.131:59597)
0810 18:51:03.290394 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:03.290457 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":86,"threads_started":1}
I20260810 18:51:03.456462 4231 raft_consensus.cc:482] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:51:03.456740 4231 raft_consensus.cc:504] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6bbcf658b984e70bdc8f1826d4f8222" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 38147 } } peers { permanent_uuid: "436302659835484193079dea06d4e462" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39573 } } peers { permanent_uuid: "7c3c35e869e641a98116d03ab65f70ca" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37577 } }
I20260810 18:51:03.458052 4231 leader_election.cc:288] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a6bbcf658b984e70bdc8f1826d4f8222 (127.18.168.131:38147), 436302659835484193079dea06d4e462 (127.18.168.130:39573)
Aug 10 18:51:03 dist-test-slave-cbsf krb5kdc[2388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387861, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for kudu/127.18.168.131@KRBTEST.COM
Aug 10 18:51:03 dist-test-slave-cbsf krb5kdc[2388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387861, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for kudu/127.18.168.130@KRBTEST.COM
I20260810 18:51:03.475138 4232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:03.458302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:39573 (local address 127.18.168.129:34141)
0810 18:51:03.458669 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:03.458684 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:03.458782 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:03.460783 (+ 2001us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:03.460785 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:03.461347 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:51:03.462243 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:03.462264 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:03.464224 (+ 1960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:03.464226 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:03.465801 (+ 1575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:03.465813 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:03.466773 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:03.466777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:03.467042 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:03.467799 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:51:03.467838 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:51:03.470095 (+ 2257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:51:03.472563 (+ 2468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:03.472566 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:03.472567 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:03.472877 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:03.473317 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:03.473319 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:03.473320 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:03.473439 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:03.474123 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:51:03.474129 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:51:03.474402 (+ 273us) client_negotiation.cc:769] Sending connection context
0810 18:51:03.474587 (+ 185us) client_negotiation.cc:240] Negotiation successful
0810 18:51:03.474944 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":476,"spinlock_wait_cycles":48768,"thread_start_us":64,"threads_started":1}
I20260810 18:51:03.475237 4234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:03.458553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:34141 (local address 127.18.168.130:39573)
0810 18:51:03.458891 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:03.458908 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:03.458938 (+ 30us) server_negotiation.cc:407] Connection header received
0810 18:51:03.459825 (+ 887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:03.459833 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:03.460310 (+ 477us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:03.460636 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:03.462441 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:03.464047 (+ 1606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:03.465929 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:03.466637 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:03.466812 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:03.470327 (+ 3515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:03.470332 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:03.470344 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:03.470371 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:03.472433 (+ 2062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:03.473023 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:03.473027 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:03.473032 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:03.473145 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:03.473571 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:03.473574 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:03.473576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:03.473993 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:03.474158 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:03.474768 (+ 610us) server_negotiation.cc:299] Negotiation successful
0810 18:51:03.475044 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":125,"threads_started":1}
I20260810 18:51:03.475248 4227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:03.458239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.131:38147 (local address 127.18.168.129:43365)
0810 18:51:03.458439 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:03.458454 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:03.458565 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:03.460739 (+ 2174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:03.460749 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:03.461347 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:51:03.462243 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:03.462264 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:03.464220 (+ 1956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:03.464224 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:03.465802 (+ 1578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:03.465813 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:03.466773 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:03.466777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:03.467041 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:03.467801 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:51:03.467825 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:51:03.469675 (+ 1850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:51:03.472352 (+ 2677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:03.472359 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:03.472375 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:03.472851 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:03.473248 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:03.473254 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:03.473256 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:03.473438 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:03.474172 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:51:03.474178 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:51:03.474402 (+ 224us) client_negotiation.cc:769] Sending connection context
0810 18:51:03.474554 (+ 152us) client_negotiation.cc:240] Negotiation successful
0810 18:51:03.475045 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":487,"spinlock_wait_cycles":26240}
I20260810 18:51:03.475185 4233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:03.458412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:43365 (local address 127.18.168.131:38147)
0810 18:51:03.458830 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:03.458857 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:03.458902 (+ 45us) server_negotiation.cc:407] Connection header received
0810 18:51:03.459825 (+ 923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:03.459833 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:03.460310 (+ 477us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:03.460636 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:03.462442 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:03.464047 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:03.465929 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:03.466637 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:03.466812 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:03.469976 (+ 3164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:03.469984 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:03.470001 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:03.470037 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:03.472184 (+ 2147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:03.472986 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:03.472993 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:03.472998 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:03.473127 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:03.473565 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:03.473568 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:03.473570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:03.473991 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:03.474158 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:03.474734 (+ 576us) server_negotiation.cc:299] Negotiation successful
0810 18:51:03.475006 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":142,"threads_started":1}
W20260810 18:51:03.477936 3781 leader_election.cc:341] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 436302659835484193079dea06d4e462 (127.18.168.130:39573): Not found: Tablet not found: 1f63aa2e2ee44e1e830847a6d5e748ae
W20260810 18:51:03.478091 3779 leader_election.cc:341] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a6bbcf658b984e70bdc8f1826d4f8222 (127.18.168.131:38147): Not found: Tablet not found: 1f63aa2e2ee44e1e830847a6d5e748ae
I20260810 18:51:03.478180 3779 leader_election.cc:302] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c3c35e869e641a98116d03ab65f70ca; no voters: 436302659835484193079dea06d4e462, a6bbcf658b984e70bdc8f1826d4f8222
I20260810 18:51:03.478425 4231 raft_consensus.cc:2726] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 18:51:03.961133 4232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:03.960845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.129:48303)
0810 18:51:03.960972 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:03.961026 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260810 18:51:04.096798 4235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:04.096314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.130:36267)
0810 18:51:04.096592 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:04.096679 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":121,"threads_started":1}
I20260810 18:51:04.291713 4236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:04.291306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.131:52817)
0810 18:51:04.291546 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:04.291598 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":70,"threads_started":1}
I20260810 18:51:04.962323 4237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:04.961920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.129:51539)
0810 18:51:04.962171 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:04.962227 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":78,"threads_started":1}
I20260810 18:51:05.097908 4238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:05.097507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.130:33339)
0810 18:51:05.097770 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:05.097825 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":74,"threads_started":1}
I20260810 18:51:05.292877 4239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:05.292416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.131:32905)
0810 18:51:05.292675 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:05.292787 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":78,"threads_started":1}
I20260810 18:51:05.494812 4240 raft_consensus.cc:482] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:51:05.494984 4240 raft_consensus.cc:504] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6bbcf658b984e70bdc8f1826d4f8222" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 38147 } } peers { permanent_uuid: "436302659835484193079dea06d4e462" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39573 } } peers { permanent_uuid: "7c3c35e869e641a98116d03ab65f70ca" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37577 } }
I20260810 18:51:05.495528 4240 leader_election.cc:288] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a6bbcf658b984e70bdc8f1826d4f8222 (127.18.168.131:38147), 436302659835484193079dea06d4e462 (127.18.168.130:39573)
W20260810 18:51:05.496416 3781 leader_election.cc:341] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 436302659835484193079dea06d4e462 (127.18.168.130:39573): Not found: Tablet not found: 1f63aa2e2ee44e1e830847a6d5e748ae
W20260810 18:51:05.496507 3779 leader_election.cc:341] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a6bbcf658b984e70bdc8f1826d4f8222 (127.18.168.131:38147): Not found: Tablet not found: 1f63aa2e2ee44e1e830847a6d5e748ae
I20260810 18:51:05.496552 3781 leader_election.cc:302] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c3c35e869e641a98116d03ab65f70ca; no voters: 436302659835484193079dea06d4e462, a6bbcf658b984e70bdc8f1826d4f8222
I20260810 18:51:05.496738 4240 raft_consensus.cc:2726] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 18:51:05.963662 4241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:05.963253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.129:37223)
0810 18:51:05.963508 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:05.963565 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":79,"threads_started":1}
I20260810 18:51:06.099077 4243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:06.098645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.130:48477)
0810 18:51:06.098913 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:06.098970 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":98,"threads_started":1}
I20260810 18:51:06.294201 4244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:06.293777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.131:37503)
0810 18:51:06.294057 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:06.294115 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":99,"threads_started":1}
I20260810 18:51:06.965304 4245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:06.964739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.129:32961)
0810 18:51:06.965126 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:06.965193 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":97,"threads_started":1}
I20260810 18:51:07.100560 4246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:07.100116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.130:41649)
0810 18:51:07.100394 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:07.100449 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":119,"threads_started":1}
I20260810 18:51:07.124475 4247 raft_consensus.cc:482] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:51:07.124657 4247 raft_consensus.cc:504] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6bbcf658b984e70bdc8f1826d4f8222" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 38147 } } peers { permanent_uuid: "436302659835484193079dea06d4e462" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39573 } } peers { permanent_uuid: "7c3c35e869e641a98116d03ab65f70ca" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37577 } }
I20260810 18:51:07.125174 4247 leader_election.cc:288] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a6bbcf658b984e70bdc8f1826d4f8222 (127.18.168.131:38147), 436302659835484193079dea06d4e462 (127.18.168.130:39573)
W20260810 18:51:07.126034 3781 leader_election.cc:341] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 436302659835484193079dea06d4e462 (127.18.168.130:39573): Not found: Tablet not found: 1f63aa2e2ee44e1e830847a6d5e748ae
W20260810 18:51:07.126145 3779 leader_election.cc:341] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a6bbcf658b984e70bdc8f1826d4f8222 (127.18.168.131:38147): Not found: Tablet not found: 1f63aa2e2ee44e1e830847a6d5e748ae
I20260810 18:51:07.126277 3779 leader_election.cc:302] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c3c35e869e641a98116d03ab65f70ca; no voters: 436302659835484193079dea06d4e462, a6bbcf658b984e70bdc8f1826d4f8222
I20260810 18:51:07.126446 4247 raft_consensus.cc:2726] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 18:51:07.295347 4248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:07.294962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.131:43933)
0810 18:51:07.295208 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:07.295262 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":74,"threads_started":1}
I20260810 18:51:07.966602 4249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:07.966127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.129:37523)
0810 18:51:07.966406 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:07.966458 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":100,"threads_started":1}
I20260810 18:51:08.101917 4250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:08.101334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.130:54823)
0810 18:51:08.101712 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:08.101786 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":114,"threads_started":1}
I20260810 18:51:08.296677 4251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:08.296158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.131:48055)
0810 18:51:08.296509 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:08.296586 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":74,"threads_started":1}
I20260810 18:51:08.967883 4252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:08.967378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.129:34143)
0810 18:51:08.967723 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:08.967784 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":75,"threads_started":1}
I20260810 18:51:09.103193 4253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:09.102721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.130:34841)
0810 18:51:09.103045 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:09.103099 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":80,"threads_started":1}
I20260810 18:51:09.297978 4254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:09.297484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.131:42059)
0810 18:51:09.297830 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:09.297882 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":105,"threads_started":1}
I20260810 18:51:09.969128 4255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:09.968635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.129:51041)
0810 18:51:09.968975 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:09.969034 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":131,"threads_started":1}
I20260810 18:51:10.104399 4256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:10.103943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.130:51687)
0810 18:51:10.104224 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:10.104297 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":79,"threads_started":1}
I20260810 18:51:10.299331 4257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:10.298868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.131:58241)
0810 18:51:10.299140 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:10.299220 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":99,"threads_started":1}
I20260810 18:51:10.970381 4258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:10.969944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.129:54485)
0810 18:51:10.970225 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:10.970282 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":97,"threads_started":1}
I20260810 18:51:11.105722 4259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:11.105295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.130:57045)
0810 18:51:11.105574 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:11.105630 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":103,"threads_started":1}
I20260810 18:51:11.300668 4260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:11.300205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.131:44931)
0810 18:51:11.300511 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:11.300572 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":95,"threads_started":1}
I20260810 18:51:11.772481 4262 raft_consensus.cc:482] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:51:11.772643 4262 raft_consensus.cc:504] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6bbcf658b984e70bdc8f1826d4f8222" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 38147 } } peers { permanent_uuid: "436302659835484193079dea06d4e462" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39573 } } peers { permanent_uuid: "7c3c35e869e641a98116d03ab65f70ca" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37577 } }
I20260810 18:51:11.773234 4262 leader_election.cc:288] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a6bbcf658b984e70bdc8f1826d4f8222 (127.18.168.131:38147), 436302659835484193079dea06d4e462 (127.18.168.130:39573)
W20260810 18:51:11.773991 3781 leader_election.cc:341] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 436302659835484193079dea06d4e462 (127.18.168.130:39573): Not found: Tablet not found: 1f63aa2e2ee44e1e830847a6d5e748ae
W20260810 18:51:11.774098 3779 leader_election.cc:341] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a6bbcf658b984e70bdc8f1826d4f8222 (127.18.168.131:38147): Not found: Tablet not found: 1f63aa2e2ee44e1e830847a6d5e748ae
I20260810 18:51:11.774230 3779 leader_election.cc:302] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c3c35e869e641a98116d03ab65f70ca; no voters: 436302659835484193079dea06d4e462, a6bbcf658b984e70bdc8f1826d4f8222
I20260810 18:51:11.774416 4262 raft_consensus.cc:2726] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 18:51:11.971848 4263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:11.971409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.129:52525)
0810 18:51:11.971683 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:11.971738 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":83,"threads_started":1}
I20260810 18:51:12.106915 4264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:12.106516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.130:45501)
0810 18:51:12.106774 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:12.106827 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":94,"threads_started":1}
I20260810 18:51:12.302062 4265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:12.301621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.131:45607)
0810 18:51:12.301907 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:12.301963 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":91,"threads_started":1}
I20260810 18:51:12.973107 4266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:12.972626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.129:59603)
0810 18:51:12.972930 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:12.972999 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":136,"threads_started":1}
I20260810 18:51:13.108157 4267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:13.107725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.130:40223)
0810 18:51:13.108001 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:13.108061 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":77,"threads_started":1}
I20260810 18:51:13.303261 4268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:13.302853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.131:33839)
0810 18:51:13.303109 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:13.303166 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":78,"threads_started":1}
I20260810 18:51:13.974462 4269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:13.974048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.129:56635)
0810 18:51:13.974303 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:13.974362 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":79,"threads_started":1}
I20260810 18:51:14.109562 4270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:14.109145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.130:56249)
0810 18:51:14.109404 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:14.109462 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":81,"threads_started":1}
I20260810 18:51:14.304484 4271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:14.303992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.131:41987)
0810 18:51:14.304325 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:14.304385 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":105,"threads_started":1}
I20260810 18:51:14.975713 4272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:14.975272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.129:55691)
0810 18:51:14.975543 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:14.975602 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":99,"threads_started":1}
I20260810 18:51:15.110805 4273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:15.110336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.130:46599)
0810 18:51:15.110647 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:15.110712 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":91,"threads_started":1}
I20260810 18:51:15.305691 4275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:15.305264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.131:51221)
0810 18:51:15.305538 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:15.305593 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":100,"threads_started":1}
I20260810 18:51:15.976986 4276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:15.976500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.129:60857)
0810 18:51:15.976826 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:15.976888 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":80,"threads_started":1}
I20260810 18:51:16.111980 4278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:16.111548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.130:52029)
0810 18:51:16.111827 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:16.111886 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":92,"threads_started":1}
I20260810 18:51:16.271376 4279 raft_consensus.cc:482] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:51:16.271539 4279 raft_consensus.cc:504] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6bbcf658b984e70bdc8f1826d4f8222" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 38147 } } peers { permanent_uuid: "436302659835484193079dea06d4e462" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39573 } } peers { permanent_uuid: "7c3c35e869e641a98116d03ab65f70ca" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 37577 } }
I20260810 18:51:16.271953 4279 leader_election.cc:288] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a6bbcf658b984e70bdc8f1826d4f8222 (127.18.168.131:38147), 436302659835484193079dea06d4e462 (127.18.168.130:39573)
W20260810 18:51:16.272729 3781 leader_election.cc:341] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 436302659835484193079dea06d4e462 (127.18.168.130:39573): Not found: Tablet not found: 1f63aa2e2ee44e1e830847a6d5e748ae
W20260810 18:51:16.272850 3779 leader_election.cc:341] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a6bbcf658b984e70bdc8f1826d4f8222 (127.18.168.131:38147): Not found: Tablet not found: 1f63aa2e2ee44e1e830847a6d5e748ae
I20260810 18:51:16.272966 3779 leader_election.cc:302] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c3c35e869e641a98116d03ab65f70ca; no voters: 436302659835484193079dea06d4e462, a6bbcf658b984e70bdc8f1826d4f8222
I20260810 18:51:16.273113 4279 raft_consensus.cc:2726] T 1f63aa2e2ee44e1e830847a6d5e748ae P 7c3c35e869e641a98116d03ab65f70ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 18:51:16.306991 4280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:16.306459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.131:33953)
0810 18:51:16.306840 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:16.306898 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":83,"threads_started":1}
I20260810 18:51:16.978360 4281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:16.977924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.129:34205)
0810 18:51:16.978208 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:16.978264 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":90,"threads_started":1}
I20260810 18:51:17.113384 4282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:17.112952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.130:48871)
0810 18:51:17.113224 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:17.113286 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":97,"threads_started":1}
I20260810 18:51:17.303148 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 3716
I20260810 18:51:17.310024 4283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:17.309425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:40999 (local address 127.18.168.131:50171)
0810 18:51:17.309828 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:17.309896 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":140,"threads_started":1}
I20260810 18:51:17.319164 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 3918
I20260810 18:51:17.327014 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 4075
2026-08-10T18:51:17Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19439 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:51:17 dist-test-slave-cbsf krb5kdc[4287](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:51:17 dist-test-slave-cbsf krb5kdc[4287](info): set up 2 sockets
Aug 10 18:51:17 dist-test-slave-cbsf krb5kdc[4287](info): commencing operation
krb5kdc: starting...
W20260810 18:51:18.460801 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:51:18 dist-test-slave-cbsf krb5kdc[4287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:51:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:51:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.168.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
I20260810 18:51:18.592060 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:41883
--webserver_interface=127.18.168.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:33555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:41883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:51:18.705232 4305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:51:18.705531 4305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:51:18.705621 4305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:51:18.709556 4305 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 18:51:18.709628 4305 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 18:51:18.709662 4305 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 18:51:18.709699 4305 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:51:18.709744 4305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:51:18.709790 4305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:51:18.709832 4305 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:51:18.709872 4305 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:51:18.715059 4305 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:33555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:41883
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:41883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:51:18.716127 4305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:51:18.717250 4305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:51:18.725947 4310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:18.725950 4313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:18.726079 4305 server_base.cc:1034] running on GCE node
W20260810 18:51:18.726248 4311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:18.726840 4305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:51:18.727864 4305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:51:18.729050 4305 hybrid_clock.cc:648] HybridClock initialized: now 1786387878729019 us; error 47 us; skew 500 ppm
Aug 10 18:51:18 dist-test-slave-cbsf krb5kdc[4287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387878, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:18.731732 4305 init.cc:377] Logged in from keytab as kudu/127.18.168.190@KRBTEST.COM (short username kudu)
I20260810 18:51:18.732591 4305 webserver.cc:460] Webserver started at http://127.18.168.190:46135/ using document root <none> and password file <none>
I20260810 18:51:18.733165 4305 fs_manager.cc:359] Metadata directory not provided
I20260810 18:51:18.733228 4305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:51:18.733392 4305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:51:18.735345 4305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "82542fe70878437fb5293cf40bd85162"
format_stamp: "Formatted at 2026-08-10 18:51:18 on dist-test-slave-cbsf"
server_key: "fccb38ac37c35ca792f1674f0133c448"
server_key_iv: "8f470552633b7aaceb5fb7f80da274da"
server_key_version: "encryptionkey@0"
I20260810 18:51:18.735854 4305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "82542fe70878437fb5293cf40bd85162"
format_stamp: "Formatted at 2026-08-10 18:51:18 on dist-test-slave-cbsf"
server_key: "fccb38ac37c35ca792f1674f0133c448"
server_key_iv: "8f470552633b7aaceb5fb7f80da274da"
server_key_version: "encryptionkey@0"
I20260810 18:51:18.739451 4305 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260810 18:51:18.741678 4320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:18.742861 4305 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:51:18.743005 4305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "82542fe70878437fb5293cf40bd85162"
format_stamp: "Formatted at 2026-08-10 18:51:18 on dist-test-slave-cbsf"
server_key: "fccb38ac37c35ca792f1674f0133c448"
server_key_iv: "8f470552633b7aaceb5fb7f80da274da"
server_key_version: "encryptionkey@0"
I20260810 18:51:18.743103 4305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:51:18.755615 4305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:51:18.758903 4305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:51:18.759106 4305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:51:18.766435 4305 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:41883
I20260810 18:51:18.766736 4379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:41883 every 8 connection(s)
I20260810 18:51:18.767433 4305 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:51:18.767712 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 4305
I20260810 18:51:18.767846 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:51:18.768218 19106 external_mini_cluster.cc:1428] Setting key d6e112861de9768db8db4d652b19ee62
I20260810 18:51:18.770694 4380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 18:51:18 dist-test-slave-cbsf krb5kdc[4287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:51:18.777297 4380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162: Bootstrap starting.
I20260810 18:51:18.779969 4380 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162: Neither blocks nor log segments found. Creating new log.
I20260810 18:51:18.780874 4380 log.cc:824] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162: Log is configured to *not* fsync() on all Append() calls
I20260810 18:51:18.782883 4383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:18.769622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57556 (local address 127.18.168.190:41883)
0810 18:51:18.770051 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:18.770076 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:18.770112 (+ 36us) server_negotiation.cc:407] Connection header received
0810 18:51:18.770799 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:18.770827 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:18.771153 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:18.771420 (+ 267us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:18.772336 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:18.773427 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:18.774585 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:18.775039 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:18.775188 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:18.777903 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:18.777909 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:18.777924 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:18.777953 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:18.779771 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:18.780219 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:18.780227 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:18.780233 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:18.780379 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:18.780636 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:18.780642 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:18.780646 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:18.781186 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:18.781380 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:18.781681 (+ 301us) server_negotiation.cc:299] Negotiation successful
0810 18:51:18.781978 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":148,"threads_started":1}
I20260810 18:51:18.783115 4380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162: No bootstrap required, opened a new log
I20260810 18:51:18.786774 4380 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82542fe70878437fb5293cf40bd85162" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41883 } }
I20260810 18:51:18.786966 4380 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:51:18.787007 4380 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82542fe70878437fb5293cf40bd85162, State: Initialized, Role: FOLLOWER
I20260810 18:51:18.787386 4380 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82542fe70878437fb5293cf40bd85162" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41883 } }
I20260810 18:51:18.787570 4380 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:51:18.787655 4380 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:51:18.787755 4380 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:51:18.788637 4380 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82542fe70878437fb5293cf40bd85162" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41883 } }
I20260810 18:51:18.789012 4380 leader_election.cc:302] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82542fe70878437fb5293cf40bd85162; no voters:
I20260810 18:51:18.789312 4380 leader_election.cc:288] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:51:18.789443 4385 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:51:18.789695 4385 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [term 1 LEADER]: Becoming Leader. State: Replica: 82542fe70878437fb5293cf40bd85162, State: Running, Role: LEADER
I20260810 18:51:18.790066 4385 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82542fe70878437fb5293cf40bd85162" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41883 } }
I20260810 18:51:18.790493 4380 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:51:18.791882 4386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82542fe70878437fb5293cf40bd85162" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82542fe70878437fb5293cf40bd85162" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41883 } } }
I20260810 18:51:18.791882 4387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82542fe70878437fb5293cf40bd85162. Latest consensus state: current_term: 1 leader_uuid: "82542fe70878437fb5293cf40bd85162" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82542fe70878437fb5293cf40bd85162" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41883 } } }
I20260810 18:51:18.792047 4387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [sys.catalog]: This master's current role is: LEADER
I20260810 18:51:18.792047 4386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [sys.catalog]: This master's current role is: LEADER
I20260810 18:51:18.792423 4398 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:51:18.795502 4398 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:51:18.801182 4398 catalog_manager.cc:1347] Generated new cluster ID: 00f351aa82bc47d084c4fd8596cfdfe6
I20260810 18:51:18.801258 4398 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:51:18.833629 4398 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:51:18.834625 4398 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:51:18.840380 4398 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162: Generated new TSK 0
I20260810 18:51:18.841075 4398 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.168.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab.
Entry for principal kudu/127.18.168.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab.
I20260810 18:51:18.879598 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.129:0
--local_ip_for_outbound_sockets=127.18.168.129
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:41883
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:33555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:51:18.998533 4407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:51:18.998806 4407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:51:18.998895 4407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:51:19.003888 4407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:51:19.003966 4407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:51:19.004081 4407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.129
I20260810 18:51:19.009645 4407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:33555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:41883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.129
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:51:19.010774 4407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:51:19.011829 4407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:51:19.021726 4412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:19.021795 4415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:19.021818 4413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:19.021989 4407 server_base.cc:1034] running on GCE node
I20260810 18:51:19.022591 4407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:51:19.023232 4407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:51:19.024446 4407 hybrid_clock.cc:648] HybridClock initialized: now 1786387879024427 us; error 50 us; skew 500 ppm
Aug 10 18:51:19 dist-test-slave-cbsf krb5kdc[4287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387879, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:19.027371 4407 init.cc:377] Logged in from keytab as kudu/127.18.168.129@KRBTEST.COM (short username kudu)
I20260810 18:51:19.028258 4407 webserver.cc:460] Webserver started at http://127.18.168.129:40433/ using document root <none> and password file <none>
I20260810 18:51:19.028889 4407 fs_manager.cc:359] Metadata directory not provided
I20260810 18:51:19.028981 4407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:51:19.029212 4407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:51:19.031211 4407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/data/instance:
uuid: "7ab2371396004cfc840d46f847dfd2f3"
format_stamp: "Formatted at 2026-08-10 18:51:19 on dist-test-slave-cbsf"
server_key: "3e3eab977583fb2a067fc976e18dc135"
server_key_iv: "38133292ddfdaec26ef612aa5eaef318"
server_key_version: "encryptionkey@0"
I20260810 18:51:19.031773 4407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance:
uuid: "7ab2371396004cfc840d46f847dfd2f3"
format_stamp: "Formatted at 2026-08-10 18:51:19 on dist-test-slave-cbsf"
server_key: "3e3eab977583fb2a067fc976e18dc135"
server_key_iv: "38133292ddfdaec26ef612aa5eaef318"
server_key_version: "encryptionkey@0"
I20260810 18:51:19.035463 4407 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260810 18:51:19.038033 4423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:19.039300 4407 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:51:19.039438 4407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/wal
uuid: "7ab2371396004cfc840d46f847dfd2f3"
format_stamp: "Formatted at 2026-08-10 18:51:19 on dist-test-slave-cbsf"
server_key: "3e3eab977583fb2a067fc976e18dc135"
server_key_iv: "38133292ddfdaec26ef612aa5eaef318"
server_key_version: "encryptionkey@0"
I20260810 18:51:19.039593 4407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:51:19.050410 4407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:51:19.053435 4407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:51:19.053661 4407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:51:19.054298 4407 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:51:19.055189 4407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:51:19.055263 4407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:19.055333 4407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:51:19.055382 4407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:19.064644 4407 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.129:39577
I20260810 18:51:19.064678 4551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.129:39577 every 8 connection(s)
I20260810 18:51:19.065622 4407 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
I20260810 18:51:19.066843 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 4407
I20260810 18:51:19.066998 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance
I20260810 18:51:19.067376 19106 external_mini_cluster.cc:1428] Setting key 141481bd5fa9d1002c55e35ccba7eb1f
Aug 10 18:51:19 dist-test-slave-cbsf krb5kdc[4287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387879, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.168.130@KRBTEST.COM; defaulting to no policy
I20260810 18:51:19.081298 4383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:19.067576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:52977 (local address 127.18.168.190:41883)
0810 18:51:19.067729 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:19.067733 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:19.068748 (+ 1015us) server_negotiation.cc:407] Connection header received
0810 18:51:19.070180 (+ 1432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:19.070187 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:19.070255 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:19.070353 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:19.071771 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.072295 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.074057 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.074288 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.074368 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:19.077449 (+ 3081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:19.077453 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:19.077456 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:19.077489 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:19.079178 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:19.079753 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:19.079757 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:19.079759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:19.079849 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:19.080154 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:19.080156 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:19.080157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:19.080409 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:19.080487 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:19.081053 (+ 566us) server_negotiation.cc:299] Negotiation successful
0810 18:51:19.081169 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.130@KRBTEST.COM" created.
I20260810 18:51:19.082324 4554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:19.068040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.129:52977)
0810 18:51:19.068538 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:19.068597 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:19.069947 (+ 1350us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:19.070513 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:19.070520 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:19.070892 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:51:19.071594 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:19.071627 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.072423 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.072430 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:19.073924 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:19.073938 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.074474 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.074479 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:19.074734 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:19.075369 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:51:19.075394 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:51:19.077267 (+ 1873us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:51:19.079298 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:19.079303 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:19.079315 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:19.079634 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:19.079931 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:19.079933 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:19.079934 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:19.080067 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:19.080510 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:51:19.080515 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:51:19.080829 (+ 314us) client_negotiation.cc:769] Sending connection context
0810 18:51:19.080989 (+ 160us) client_negotiation.cc:240] Negotiation successful
0810 18:51:19.081263 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":115,"threads_started":1}
I20260810 18:51:19.083696 4552 heartbeater.cc:347] Connected to a master server at 127.18.168.190:41883
I20260810 18:51:19.083966 4552 heartbeater.cc:464] Registering TS with master...
I20260810 18:51:19.084501 4552 heartbeater.cc:511] Master 127.18.168.190:41883 requested a full tablet report, sending...
I20260810 18:51:19.086247 4339 ts_manager.cc:194] Registered new tserver with Master: 7ab2371396004cfc840d46f847dfd2f3 (127.18.168.129:39577)
I20260810 18:51:19.087967 4339 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.129@KRBTEST.COM'} at 127.18.168.129:52977
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab.
Entry for principal kudu/127.18.168.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab.
I20260810 18:51:19.099439 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.130:0
--local_ip_for_outbound_sockets=127.18.168.130
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:41883
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:33555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:51:19.212786 4558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:51:19.213063 4558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:51:19.213104 4558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:51:19.217123 4558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:51:19.217200 4558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:51:19.217258 4558 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.130
I20260810 18:51:19.222685 4558 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:33555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:41883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4558
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.130
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:51:19.223757 4558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:51:19.224792 4558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:51:19.234468 4566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:19.234555 4558 server_base.cc:1034] running on GCE node
W20260810 18:51:19.234485 4565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:19.234488 4568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:19.235126 4558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:51:19.235736 4558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:51:19.236907 4558 hybrid_clock.cc:648] HybridClock initialized: now 1786387879236895 us; error 27 us; skew 500 ppm
Aug 10 18:51:19 dist-test-slave-cbsf krb5kdc[4287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387879, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:19.239629 4558 init.cc:377] Logged in from keytab as kudu/127.18.168.130@KRBTEST.COM (short username kudu)
I20260810 18:51:19.240463 4558 webserver.cc:460] Webserver started at http://127.18.168.130:44477/ using document root <none> and password file <none>
I20260810 18:51:19.241081 4558 fs_manager.cc:359] Metadata directory not provided
I20260810 18:51:19.241150 4558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:51:19.241389 4558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:51:19.243438 4558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/data/instance:
uuid: "a5f555d7ab0d43248673885b404448ca"
format_stamp: "Formatted at 2026-08-10 18:51:19 on dist-test-slave-cbsf"
server_key: "04b6d50707eb5d13c4bae123288f2742"
server_key_iv: "4d89f223ce06e066b64872d6da3489e2"
server_key_version: "encryptionkey@0"
I20260810 18:51:19.244045 4558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance:
uuid: "a5f555d7ab0d43248673885b404448ca"
format_stamp: "Formatted at 2026-08-10 18:51:19 on dist-test-slave-cbsf"
server_key: "04b6d50707eb5d13c4bae123288f2742"
server_key_iv: "4d89f223ce06e066b64872d6da3489e2"
server_key_version: "encryptionkey@0"
I20260810 18:51:19.247840 4558 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:51:19.250211 4577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:19.251389 4558 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:51:19.251523 4558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/wal
uuid: "a5f555d7ab0d43248673885b404448ca"
format_stamp: "Formatted at 2026-08-10 18:51:19 on dist-test-slave-cbsf"
server_key: "04b6d50707eb5d13c4bae123288f2742"
server_key_iv: "4d89f223ce06e066b64872d6da3489e2"
server_key_version: "encryptionkey@0"
I20260810 18:51:19.251638 4558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:51:19.270892 4558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:51:19.273808 4558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:51:19.274008 4558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:51:19.274592 4558 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:51:19.275419 4558 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:51:19.275490 4558 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:19.275558 4558 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:51:19.275601 4558 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:19.284775 4558 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.130:39705
I20260810 18:51:19.284827 4709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.130:39705 every 8 connection(s)
I20260810 18:51:19.285631 4558 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
I20260810 18:51:19.286592 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 4558
I20260810 18:51:19.286703 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance
I20260810 18:51:19.287022 19106 external_mini_cluster.cc:1428] Setting key 2e9cff2d2dc17739ee90cb0902a50d68
Aug 10 18:51:19 dist-test-slave-cbsf krb5kdc[4287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387879, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.168.131@KRBTEST.COM; defaulting to no policy
I20260810 18:51:19.302064 4383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:19.287549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:57975 (local address 127.18.168.190:41883)
0810 18:51:19.287700 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:19.287705 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:19.288742 (+ 1037us) server_negotiation.cc:407] Connection header received
0810 18:51:19.290745 (+ 2003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:19.290748 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:19.290790 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:19.290859 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:19.292312 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.293129 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.295026 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.295240 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.295322 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:19.298216 (+ 2894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:19.298220 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:19.298222 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:19.298256 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:19.299939 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:19.300642 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:19.300652 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:19.300654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:19.300799 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:19.301113 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:19.301116 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:19.301117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:19.301308 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:19.301389 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:19.301840 (+ 451us) server_negotiation.cc:299] Negotiation successful
0810 18:51:19.301944 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.131@KRBTEST.COM" created.
I20260810 18:51:19.302917 4712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:19.287975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.130:57975)
0810 18:51:19.288534 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:19.288612 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:19.290521 (+ 1909us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:19.291051 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:19.291064 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:19.291583 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:51:19.292164 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:19.292180 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.293277 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.293284 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:19.294870 (+ 1586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:19.294886 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.295411 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.295416 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:19.295679 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:19.296309 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:51:19.296336 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:51:19.298048 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:51:19.300068 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:19.300076 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:19.300090 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:19.300514 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:19.300903 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:19.300905 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:19.300907 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:19.301032 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:19.301400 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:51:19.301404 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:51:19.301632 (+ 228us) client_negotiation.cc:769] Sending connection context
0810 18:51:19.301781 (+ 149us) client_negotiation.cc:240] Negotiation successful
0810 18:51:19.302016 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":140,"threads_started":1}
I20260810 18:51:19.304090 4710 heartbeater.cc:347] Connected to a master server at 127.18.168.190:41883
I20260810 18:51:19.304355 4710 heartbeater.cc:464] Registering TS with master...
I20260810 18:51:19.304963 4710 heartbeater.cc:511] Master 127.18.168.190:41883 requested a full tablet report, sending...
I20260810 18:51:19.306085 4339 ts_manager.cc:194] Registered new tserver with Master: a5f555d7ab0d43248673885b404448ca (127.18.168.130:39705)
I20260810 18:51:19.306661 4339 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.130@KRBTEST.COM'} at 127.18.168.130:57975
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab.
Entry for principal kudu/127.18.168.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab.
I20260810 18:51:19.319113 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.131:0
--local_ip_for_outbound_sockets=127.18.168.131
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:41883
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:33555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:51:19.433516 4715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:51:19.433799 4715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:51:19.433888 4715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:51:19.438328 4715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:51:19.438410 4715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:51:19.438508 4715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.131
I20260810 18:51:19.444424 4715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:33555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:41883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.131
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:51:19.445531 4715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:51:19.446691 4715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:51:19.457122 4721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:19.457134 4724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:19.457320 4722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:19.457360 4715 server_base.cc:1034] running on GCE node
I20260810 18:51:19.457895 4715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:51:19.458487 4715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:51:19.459633 4715 hybrid_clock.cc:648] HybridClock initialized: now 1786387879459602 us; error 46 us; skew 500 ppm
Aug 10 18:51:19 dist-test-slave-cbsf krb5kdc[4287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387879, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:19.462332 4715 init.cc:377] Logged in from keytab as kudu/127.18.168.131@KRBTEST.COM (short username kudu)
I20260810 18:51:19.463155 4715 webserver.cc:460] Webserver started at http://127.18.168.131:42121/ using document root <none> and password file <none>
I20260810 18:51:19.463701 4715 fs_manager.cc:359] Metadata directory not provided
I20260810 18:51:19.463789 4715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:51:19.464032 4715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:51:19.466180 4715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/data/instance:
uuid: "89fb620a5220480c9887a6a0d6827f19"
format_stamp: "Formatted at 2026-08-10 18:51:19 on dist-test-slave-cbsf"
server_key: "8e2ec631dcc193d64d1d723ed5621860"
server_key_iv: "13c7ef17bdfe814e892f79c8a14fc294"
server_key_version: "encryptionkey@0"
I20260810 18:51:19.466732 4715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance:
uuid: "89fb620a5220480c9887a6a0d6827f19"
format_stamp: "Formatted at 2026-08-10 18:51:19 on dist-test-slave-cbsf"
server_key: "8e2ec631dcc193d64d1d723ed5621860"
server_key_iv: "13c7ef17bdfe814e892f79c8a14fc294"
server_key_version: "encryptionkey@0"
I20260810 18:51:19.470337 4715 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260810 18:51:19.472592 4731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:19.473807 4715 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:51:19.473935 4715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/wal
uuid: "89fb620a5220480c9887a6a0d6827f19"
format_stamp: "Formatted at 2026-08-10 18:51:19 on dist-test-slave-cbsf"
server_key: "8e2ec631dcc193d64d1d723ed5621860"
server_key_iv: "13c7ef17bdfe814e892f79c8a14fc294"
server_key_version: "encryptionkey@0"
I20260810 18:51:19.474041 4715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:51:19.489290 4715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:51:19.492393 4715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:51:19.492592 4715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:51:19.493206 4715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:51:19.494047 4715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:51:19.494120 4715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:19.494208 4715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:51:19.494259 4715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:19.503031 4715 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.131:43805
I20260810 18:51:19.503050 4870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.131:43805 every 8 connection(s)
I20260810 18:51:19.503906 4715 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
I20260810 18:51:19.505848 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 4715
I20260810 18:51:19.505976 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance
I20260810 18:51:19.506311 19106 external_mini_cluster.cc:1428] Setting key a404ec1bf6ebb9fc67375814ff48324a
Aug 10 18:51:19 dist-test-slave-cbsf krb5kdc[4287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387879, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:51:19.517745 4383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:19.505758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:52929 (local address 127.18.168.190:41883)
0810 18:51:19.505896 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:19.505899 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:19.507033 (+ 1134us) server_negotiation.cc:407] Connection header received
0810 18:51:19.508745 (+ 1712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:19.508748 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:19.508809 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:19.508878 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:19.509996 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.510519 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.511624 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.511848 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.511929 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:19.514278 (+ 2349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:19.514281 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:19.514283 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:19.514307 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:19.515766 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:19.516242 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:19.516246 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:19.516247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:19.516331 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:19.516662 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:19.516665 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:19.516666 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:19.516936 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:19.517027 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:19.517495 (+ 468us) server_negotiation.cc:299] Negotiation successful
0810 18:51:19.517601 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 18:51:19.518536 4873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:19.506242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.131:52929)
0810 18:51:19.506853 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:19.506913 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:19.508495 (+ 1582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:19.509034 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:19.509040 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:19.509397 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:51:19.509877 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:19.509889 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.510627 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.510630 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:19.511522 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:19.511531 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.511974 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.511977 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:19.512110 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:19.512635 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:51:19.512651 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:51:19.514129 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:51:19.515869 (+ 1740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:19.515873 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:19.515885 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:19.516128 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:19.516419 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:19.516421 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:19.516422 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:19.516544 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:19.517034 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:51:19.517039 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:51:19.517279 (+ 240us) client_negotiation.cc:769] Sending connection context
0810 18:51:19.517434 (+ 155us) client_negotiation.cc:240] Negotiation successful
0810 18:51:19.517685 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":156,"threads_started":1}
I20260810 18:51:19.519858 4871 heartbeater.cc:347] Connected to a master server at 127.18.168.190:41883
I20260810 18:51:19.520139 4871 heartbeater.cc:464] Registering TS with master...
I20260810 18:51:19.520680 4871 heartbeater.cc:511] Master 127.18.168.190:41883 requested a full tablet report, sending...
I20260810 18:51:19.521741 4339 ts_manager.cc:194] Registered new tserver with Master: 89fb620a5220480c9887a6a0d6827f19 (127.18.168.131:43805)
I20260810 18:51:19.522260 4339 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.131@KRBTEST.COM'} at 127.18.168.131:52929
I20260810 18:51:19.532089 19106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:51:19.541128 4383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:19.534220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57568 (local address 127.18.168.190:41883)
0810 18:51:19.534350 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:19.534354 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:19.534548 (+ 194us) server_negotiation.cc:407] Connection header received
0810 18:51:19.534650 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:19.534652 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:19.534692 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:19.534758 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:19.535436 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.535937 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.536791 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.537009 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.537087 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:19.538442 (+ 1355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:19.538446 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:19.538448 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:19.538469 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:19.539680 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:19.540048 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:19.540050 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:19.540052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:19.540130 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:19.540356 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:19.540359 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:19.540360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:19.540557 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:19.540662 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:19.540915 (+ 253us) server_negotiation.cc:299] Negotiation successful
0810 18:51:19.541006 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 18:51:19.544083 4339 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57568:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:51:19.547359 4339 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:51:19.563257 4885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:19.556613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:39577 (local address 127.0.0.1:34930)
0810 18:51:19.557092 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:19.557139 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:19.557283 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:19.558819 (+ 1536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:19.558822 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:19.558846 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:51:19.558937 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:19.558944 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.560047 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.560052 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:19.561795 (+ 1743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:19.561806 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.562612 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.562616 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:19.562722 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:19.562759 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 18:51:19.562967 (+ 208us) client_negotiation.cc:240] Negotiation successful
0810 18:51:19.563094 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":213,"thread_start_us":239,"threads_started":1}
I20260810 18:51:19.563268 4884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:19.556728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34930 (local address 127.18.168.129:39577)
0810 18:51:19.558330 (+ 1602us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:19.558364 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:19.558391 (+ 27us) server_negotiation.cc:407] Connection header received
0810 18:51:19.558473 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:19.558478 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:19.558660 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:19.558848 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:51:19.559036 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.559897 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.561926 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.562471 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.562605 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:19.562788 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:19.562991 (+ 203us) server_negotiation.cc:299] Negotiation successful
0810 18:51:19.563093 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1489,"thread_start_us":111,"threads_started":1}
I20260810 18:51:19.563544 4886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:19.557424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.131:43805 (local address 127.0.0.1:54032)
0810 18:51:19.557937 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:19.557960 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:19.558041 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:19.559130 (+ 1089us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:19.559132 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:19.559150 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:51:19.559237 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:19.559242 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.560455 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.560460 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:19.561681 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:19.561691 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.563021 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.563025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:19.563213 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:19.563252 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 18:51:19.563324 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 18:51:19.563398 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"mutex_wait_us":127,"thread_start_us":137,"threads_started":1}
I20260810 18:51:19.563651 4887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:19.557498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52602 (local address 127.18.168.130:39705)
0810 18:51:19.557868 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:19.557893 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:19.557918 (+ 25us) server_negotiation.cc:407] Connection header received
0810 18:51:19.558006 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:19.558011 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:19.558242 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:19.558424 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:51:19.558800 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.560017 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.561878 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.562649 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.562797 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:19.562968 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:19.563394 (+ 426us) server_negotiation.cc:299] Negotiation successful
0810 18:51:19.563493 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":166,"threads_started":1}
I20260810 18:51:19.563973 4888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:19.557663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54032 (local address 127.18.168.131:43805)
0810 18:51:19.558695 (+ 1032us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:19.558705 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:19.558748 (+ 43us) server_negotiation.cc:407] Connection header received
0810 18:51:19.558822 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:19.558827 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:19.559003 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:19.559160 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:51:19.559319 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.560326 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.561809 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.562649 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.562797 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:19.562950 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:19.563719 (+ 769us) server_negotiation.cc:299] Negotiation successful
0810 18:51:19.563839 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":946,"thread_start_us":197,"threads_started":1}
I20260810 18:51:19.564054 4883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:19.556640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:39705 (local address 127.0.0.1:52602)
0810 18:51:19.557148 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:19.557177 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:19.557292 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:19.558487 (+ 1195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:19.558493 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:19.558518 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:51:19.558654 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:19.558661 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.560154 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.560157 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:19.561700 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:19.561712 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.563021 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.563024 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:19.563208 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:19.563250 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 18:51:19.563838 (+ 588us) client_negotiation.cc:240] Negotiation successful
0810 18:51:19.563910 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"mutex_wait_us":178,"thread_start_us":120,"threads_started":1}
I20260810 18:51:19.566951 4634 tablet_service.cc:1467] Processing CreateTablet for tablet 45015a60705843d2a86237efdd42014a (DEFAULT_TABLE table=test-table [id=2295e10906f944d8a4589e525a0c6444]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:51:19.566964 4792 tablet_service.cc:1467] Processing CreateTablet for tablet 45015a60705843d2a86237efdd42014a (DEFAULT_TABLE table=test-table [id=2295e10906f944d8a4589e525a0c6444]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:51:19.566967 4482 tablet_service.cc:1467] Processing CreateTablet for tablet 45015a60705843d2a86237efdd42014a (DEFAULT_TABLE table=test-table [id=2295e10906f944d8a4589e525a0c6444]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:51:19.568005 4634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45015a60705843d2a86237efdd42014a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:51:19.568025 4482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45015a60705843d2a86237efdd42014a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:51:19.568034 4792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45015a60705843d2a86237efdd42014a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:51:19.574805 4889 tablet_bootstrap.cc:492] T 45015a60705843d2a86237efdd42014a P a5f555d7ab0d43248673885b404448ca: Bootstrap starting.
I20260810 18:51:19.575953 4890 tablet_bootstrap.cc:492] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3: Bootstrap starting.
I20260810 18:51:19.575979 4893 tablet_bootstrap.cc:492] T 45015a60705843d2a86237efdd42014a P 89fb620a5220480c9887a6a0d6827f19: Bootstrap starting.
I20260810 18:51:19.577749 4889 tablet_bootstrap.cc:654] T 45015a60705843d2a86237efdd42014a P a5f555d7ab0d43248673885b404448ca: Neither blocks nor log segments found. Creating new log.
I20260810 18:51:19.578219 4890 tablet_bootstrap.cc:654] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3: Neither blocks nor log segments found. Creating new log.
I20260810 18:51:19.578219 4893 tablet_bootstrap.cc:654] T 45015a60705843d2a86237efdd42014a P 89fb620a5220480c9887a6a0d6827f19: Neither blocks nor log segments found. Creating new log.
I20260810 18:51:19.578704 4889 log.cc:824] T 45015a60705843d2a86237efdd42014a P a5f555d7ab0d43248673885b404448ca: Log is configured to *not* fsync() on all Append() calls
I20260810 18:51:19.579628 4890 log.cc:824] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3: Log is configured to *not* fsync() on all Append() calls
I20260810 18:51:19.579627 4893 log.cc:824] T 45015a60705843d2a86237efdd42014a P 89fb620a5220480c9887a6a0d6827f19: Log is configured to *not* fsync() on all Append() calls
I20260810 18:51:19.581298 4889 tablet_bootstrap.cc:492] T 45015a60705843d2a86237efdd42014a P a5f555d7ab0d43248673885b404448ca: No bootstrap required, opened a new log
I20260810 18:51:19.581393 4890 tablet_bootstrap.cc:492] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3: No bootstrap required, opened a new log
I20260810 18:51:19.581535 4889 ts_tablet_manager.cc:1397] T 45015a60705843d2a86237efdd42014a P a5f555d7ab0d43248673885b404448ca: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260810 18:51:19.581610 4893 tablet_bootstrap.cc:492] T 45015a60705843d2a86237efdd42014a P 89fb620a5220480c9887a6a0d6827f19: No bootstrap required, opened a new log
I20260810 18:51:19.581581 4890 ts_tablet_manager.cc:1397] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260810 18:51:19.581789 4893 ts_tablet_manager.cc:1397] T 45015a60705843d2a86237efdd42014a P 89fb620a5220480c9887a6a0d6827f19: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260810 18:51:19.584484 4890 raft_consensus.cc:348] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab2371396004cfc840d46f847dfd2f3" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39577 } } peers { permanent_uuid: "a5f555d7ab0d43248673885b404448ca" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39705 } } peers { permanent_uuid: "89fb620a5220480c9887a6a0d6827f19" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43805 } }
I20260810 18:51:19.584676 4890 raft_consensus.cc:374] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:51:19.584765 4890 raft_consensus.cc:729] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ab2371396004cfc840d46f847dfd2f3, State: Initialized, Role: FOLLOWER
I20260810 18:51:19.585232 4890 consensus_queue.cc:260] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab2371396004cfc840d46f847dfd2f3" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39577 } } peers { permanent_uuid: "a5f555d7ab0d43248673885b404448ca" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39705 } } peers { permanent_uuid: "89fb620a5220480c9887a6a0d6827f19" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43805 } }
I20260810 18:51:19.585521 4893 raft_consensus.cc:348] T 45015a60705843d2a86237efdd42014a P 89fb620a5220480c9887a6a0d6827f19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab2371396004cfc840d46f847dfd2f3" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39577 } } peers { permanent_uuid: "a5f555d7ab0d43248673885b404448ca" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39705 } } peers { permanent_uuid: "89fb620a5220480c9887a6a0d6827f19" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43805 } }
I20260810 18:51:19.585784 4893 raft_consensus.cc:374] T 45015a60705843d2a86237efdd42014a P 89fb620a5220480c9887a6a0d6827f19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:51:19.585837 4893 raft_consensus.cc:729] T 45015a60705843d2a86237efdd42014a P 89fb620a5220480c9887a6a0d6827f19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89fb620a5220480c9887a6a0d6827f19, State: Initialized, Role: FOLLOWER
I20260810 18:51:19.585907 4552 heartbeater.cc:503] Master 127.18.168.190:41883 was elected leader, sending a full tablet report...
I20260810 18:51:19.586192 4890 ts_tablet_manager.cc:1428] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20260810 18:51:19.586349 4893 consensus_queue.cc:260] T 45015a60705843d2a86237efdd42014a P 89fb620a5220480c9887a6a0d6827f19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab2371396004cfc840d46f847dfd2f3" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39577 } } peers { permanent_uuid: "a5f555d7ab0d43248673885b404448ca" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39705 } } peers { permanent_uuid: "89fb620a5220480c9887a6a0d6827f19" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43805 } }
I20260810 18:51:19.587051 4871 heartbeater.cc:503] Master 127.18.168.190:41883 was elected leader, sending a full tablet report...
I20260810 18:51:19.587393 4893 ts_tablet_manager.cc:1428] T 45015a60705843d2a86237efdd42014a P 89fb620a5220480c9887a6a0d6827f19: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260810 18:51:19.585521 4889 raft_consensus.cc:348] T 45015a60705843d2a86237efdd42014a P a5f555d7ab0d43248673885b404448ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab2371396004cfc840d46f847dfd2f3" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39577 } } peers { permanent_uuid: "a5f555d7ab0d43248673885b404448ca" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39705 } } peers { permanent_uuid: "89fb620a5220480c9887a6a0d6827f19" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43805 } }
I20260810 18:51:19.589025 4889 raft_consensus.cc:374] T 45015a60705843d2a86237efdd42014a P a5f555d7ab0d43248673885b404448ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:51:19.589094 4889 raft_consensus.cc:729] T 45015a60705843d2a86237efdd42014a P a5f555d7ab0d43248673885b404448ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5f555d7ab0d43248673885b404448ca, State: Initialized, Role: FOLLOWER
I20260810 18:51:19.589541 4889 consensus_queue.cc:260] T 45015a60705843d2a86237efdd42014a P a5f555d7ab0d43248673885b404448ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab2371396004cfc840d46f847dfd2f3" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39577 } } peers { permanent_uuid: "a5f555d7ab0d43248673885b404448ca" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39705 } } peers { permanent_uuid: "89fb620a5220480c9887a6a0d6827f19" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43805 } }
I20260810 18:51:19.590896 4710 heartbeater.cc:503] Master 127.18.168.190:41883 was elected leader, sending a full tablet report...
I20260810 18:51:19.591105 4889 ts_tablet_manager.cc:1428] T 45015a60705843d2a86237efdd42014a P a5f555d7ab0d43248673885b404448ca: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
W20260810 18:51:19.754906 4872 tablet.cc:2369] T 45015a60705843d2a86237efdd42014a P 89fb620a5220480c9887a6a0d6827f19: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:51:19.786741 4711 tablet.cc:2369] T 45015a60705843d2a86237efdd42014a P a5f555d7ab0d43248673885b404448ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:51:19.817220 4553 tablet.cc:2369] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:51:19.842983 4898 raft_consensus.cc:482] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:51:19.843222 4898 raft_consensus.cc:504] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab2371396004cfc840d46f847dfd2f3" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39577 } } peers { permanent_uuid: "a5f555d7ab0d43248673885b404448ca" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39705 } } peers { permanent_uuid: "89fb620a5220480c9887a6a0d6827f19" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43805 } }
I20260810 18:51:19.844482 4898 leader_election.cc:288] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5f555d7ab0d43248673885b404448ca (127.18.168.130:39705), 89fb620a5220480c9887a6a0d6827f19 (127.18.168.131:43805)
I20260810 18:51:19.848991 4901 raft_consensus.cc:482] T 45015a60705843d2a86237efdd42014a P 89fb620a5220480c9887a6a0d6827f19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:51:19.849192 4901 raft_consensus.cc:504] T 45015a60705843d2a86237efdd42014a P 89fb620a5220480c9887a6a0d6827f19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab2371396004cfc840d46f847dfd2f3" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39577 } } peers { permanent_uuid: "a5f555d7ab0d43248673885b404448ca" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39705 } } peers { permanent_uuid: "89fb620a5220480c9887a6a0d6827f19" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43805 } }
I20260810 18:51:19.849856 4888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:19.844927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:40675 (local address 127.18.168.131:43805)
0810 18:51:19.845062 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:19.845067 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:19.845353 (+ 286us) server_negotiation.cc:407] Connection header received
0810 18:51:19.845524 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:19.845526 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:19.845568 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:19.845669 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:51:19.846085 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.846787 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.848458 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.849096 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.849217 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:19.849270 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:19.849657 (+ 387us) server_negotiation.cc:299] Negotiation successful
0810 18:51:19.849745 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 18:51:19.849861 4887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:19.844858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:42711 (local address 127.18.168.130:39705)
0810 18:51:19.845014 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:19.845020 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:19.845285 (+ 265us) server_negotiation.cc:407] Connection header received
0810 18:51:19.845478 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:19.845483 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:19.845546 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:19.845604 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:51:19.845981 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.846759 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.848458 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.849096 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.849217 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:19.849270 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:19.849684 (+ 414us) server_negotiation.cc:299] Negotiation successful
0810 18:51:19.849764 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 18:51:19.849880 4903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:19.844776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.131:43805 (local address 127.18.168.129:40675)
0810 18:51:19.845276 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:19.845287 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:19.845409 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:19.845697 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:19.845700 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:19.845739 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:51:19.845942 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:19.845951 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.846906 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.846909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:19.848315 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:19.848322 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.849331 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.849335 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:19.849505 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:19.849547 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 18:51:19.849626 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 18:51:19.849711 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"mutex_wait_us":139,"thread_start_us":137,"threads_started":1}
I20260810 18:51:19.850224 4904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:19.844754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:39705 (local address 127.18.168.129:42711)
0810 18:51:19.845175 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:19.845190 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:19.845328 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:19.845639 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:19.845644 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:19.845664 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:51:19.845862 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:19.845868 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.846865 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.846871 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:19.848311 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:19.848322 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.849331 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.849335 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:19.849508 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:19.849551 (+ 43us) client_negotiation.cc:769] Sending connection context
0810 18:51:19.849995 (+ 444us) client_negotiation.cc:240] Negotiation successful
0810 18:51:19.850070 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"mutex_wait_us":160,"spinlock_wait_cycles":33152,"thread_start_us":149,"threads_started":1}
I20260810 18:51:19.850526 4901 leader_election.cc:288] T 45015a60705843d2a86237efdd42014a P 89fb620a5220480c9887a6a0d6827f19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ab2371396004cfc840d46f847dfd2f3 (127.18.168.129:39577), a5f555d7ab0d43248673885b404448ca (127.18.168.130:39705)
I20260810 18:51:19.851060 4657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45015a60705843d2a86237efdd42014a" candidate_uuid: "7ab2371396004cfc840d46f847dfd2f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5f555d7ab0d43248673885b404448ca" is_pre_election: true
I20260810 18:51:19.851168 4815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45015a60705843d2a86237efdd42014a" candidate_uuid: "7ab2371396004cfc840d46f847dfd2f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89fb620a5220480c9887a6a0d6827f19" is_pre_election: true
I20260810 18:51:19.851365 4815 raft_consensus.cc:2454] T 45015a60705843d2a86237efdd42014a P 89fb620a5220480c9887a6a0d6827f19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ab2371396004cfc840d46f847dfd2f3 in term 0.
I20260810 18:51:19.851380 4657 raft_consensus.cc:2454] T 45015a60705843d2a86237efdd42014a P a5f555d7ab0d43248673885b404448ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ab2371396004cfc840d46f847dfd2f3 in term 0.
I20260810 18:51:19.851864 4426 leader_election.cc:302] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ab2371396004cfc840d46f847dfd2f3, 89fb620a5220480c9887a6a0d6827f19; no voters:
I20260810 18:51:19.852238 4898 raft_consensus.cc:2781] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:51:19.852351 4898 raft_consensus.cc:482] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:51:19.852401 4898 raft_consensus.cc:3037] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:51:19.853881 4898 raft_consensus.cc:504] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab2371396004cfc840d46f847dfd2f3" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39577 } } peers { permanent_uuid: "a5f555d7ab0d43248673885b404448ca" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39705 } } peers { permanent_uuid: "89fb620a5220480c9887a6a0d6827f19" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43805 } }
I20260810 18:51:19.854270 4898 leader_election.cc:288] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3 [CANDIDATE]: Term 1 election: Requested vote from peers a5f555d7ab0d43248673885b404448ca (127.18.168.130:39705), 89fb620a5220480c9887a6a0d6827f19 (127.18.168.131:43805)
I20260810 18:51:19.854624 4657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45015a60705843d2a86237efdd42014a" candidate_uuid: "7ab2371396004cfc840d46f847dfd2f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5f555d7ab0d43248673885b404448ca"
I20260810 18:51:19.854761 4815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45015a60705843d2a86237efdd42014a" candidate_uuid: "7ab2371396004cfc840d46f847dfd2f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89fb620a5220480c9887a6a0d6827f19"
I20260810 18:51:19.854786 4657 raft_consensus.cc:3037] T 45015a60705843d2a86237efdd42014a P a5f555d7ab0d43248673885b404448ca [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:51:19.854885 4815 raft_consensus.cc:3037] T 45015a60705843d2a86237efdd42014a P 89fb620a5220480c9887a6a0d6827f19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:51:19.855044 4873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:19.850653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:39577 (local address 127.18.168.131:49283)
0810 18:51:19.851041 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:19.851056 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:19.851132 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:19.851377 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:19.851379 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:19.851402 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:51:19.851508 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:19.851514 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.852836 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.852839 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:19.853715 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:19.853729 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.854566 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.854571 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:19.854703 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:19.854726 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 18:51:19.854801 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 18:51:19.854876 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319}
I20260810 18:51:19.855137 4884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:19.851022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:49283 (local address 127.18.168.129:39577)
0810 18:51:19.851131 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:19.851135 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:19.851149 (+ 14us) server_negotiation.cc:407] Connection header received
0810 18:51:19.851236 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:19.851239 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:19.851279 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:19.851350 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:51:19.851628 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.852285 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.853819 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.854337 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.854439 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:19.854485 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:19.854940 (+ 455us) server_negotiation.cc:299] Negotiation successful
0810 18:51:19.855012 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 18:51:19.855737 4504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45015a60705843d2a86237efdd42014a" candidate_uuid: "89fb620a5220480c9887a6a0d6827f19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ab2371396004cfc840d46f847dfd2f3" is_pre_election: true
I20260810 18:51:19.855984 4504 raft_consensus.cc:2379] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 89fb620a5220480c9887a6a0d6827f19 in current term 1: Already voted for candidate 7ab2371396004cfc840d46f847dfd2f3 in this term.
I20260810 18:51:19.856236 4657 raft_consensus.cc:2454] T 45015a60705843d2a86237efdd42014a P a5f555d7ab0d43248673885b404448ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ab2371396004cfc840d46f847dfd2f3 in term 1.
I20260810 18:51:19.856328 4815 raft_consensus.cc:2454] T 45015a60705843d2a86237efdd42014a P 89fb620a5220480c9887a6a0d6827f19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ab2371396004cfc840d46f847dfd2f3 in term 1.
I20260810 18:51:19.856717 4427 leader_election.cc:302] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ab2371396004cfc840d46f847dfd2f3, a5f555d7ab0d43248673885b404448ca; no voters:
I20260810 18:51:19.857048 4898 raft_consensus.cc:2781] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:51:19.857334 4898 raft_consensus.cc:686] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3 [term 1 LEADER]: Becoming Leader. State: Replica: 7ab2371396004cfc840d46f847dfd2f3, State: Running, Role: LEADER
I20260810 18:51:19.857681 4898 consensus_queue.cc:237] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab2371396004cfc840d46f847dfd2f3" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39577 } } peers { permanent_uuid: "a5f555d7ab0d43248673885b404448ca" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39705 } } peers { permanent_uuid: "89fb620a5220480c9887a6a0d6827f19" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43805 } }
I20260810 18:51:19.858103 4887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:19.851703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:40701 (local address 127.18.168.130:39705)
0810 18:51:19.851808 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:19.851812 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:19.852103 (+ 291us) server_negotiation.cc:407] Connection header received
0810 18:51:19.852238 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:19.852241 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:19.852289 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:19.852355 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:51:19.852826 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.853464 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.856882 (+ 3418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.857470 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.857578 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:19.857619 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:19.857945 (+ 326us) server_negotiation.cc:299] Negotiation successful
0810 18:51:19.858008 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 18:51:19.858135 4905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:19.850726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:39705 (local address 127.18.168.131:40701)
0810 18:51:19.851999 (+ 1273us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:19.852013 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:19.852111 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:19.852502 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:19.852506 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:19.852521 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:51:19.852664 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:19.852671 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.853617 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.853621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:19.856679 (+ 3058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:19.856731 (+ 52us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.857700 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.857703 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:19.857798 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:19.857843 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 18:51:19.857915 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 18:51:19.857982 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1213,"thread_start_us":82,"threads_started":1}
I20260810 18:51:19.858558 4657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45015a60705843d2a86237efdd42014a" candidate_uuid: "89fb620a5220480c9887a6a0d6827f19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5f555d7ab0d43248673885b404448ca" is_pre_election: true
I20260810 18:51:19.858736 4657 raft_consensus.cc:2379] T 45015a60705843d2a86237efdd42014a P a5f555d7ab0d43248673885b404448ca [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 89fb620a5220480c9887a6a0d6827f19 in current term 1: Already voted for candidate 7ab2371396004cfc840d46f847dfd2f3 in this term.
I20260810 18:51:19.859165 4735 leader_election.cc:302] T 45015a60705843d2a86237efdd42014a P 89fb620a5220480c9887a6a0d6827f19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 89fb620a5220480c9887a6a0d6827f19; no voters: 7ab2371396004cfc840d46f847dfd2f3, a5f555d7ab0d43248673885b404448ca
I20260810 18:51:19.859432 4901 raft_consensus.cc:2726] T 45015a60705843d2a86237efdd42014a P 89fb620a5220480c9887a6a0d6827f19 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 18:51:19.861223 4338 catalog_manager.cc:5495] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ab2371396004cfc840d46f847dfd2f3 (127.18.168.129). New cstate: current_term: 1 leader_uuid: "7ab2371396004cfc840d46f847dfd2f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab2371396004cfc840d46f847dfd2f3" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a5f555d7ab0d43248673885b404448ca" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89fb620a5220480c9887a6a0d6827f19" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43805 } health_report { overall_health: UNKNOWN } } }
I20260810 18:51:19.926456 4884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:19.922548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34944 (local address 127.18.168.129:39577)
0810 18:51:19.922707 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:19.922712 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:19.922729 (+ 17us) server_negotiation.cc:407] Connection header received
0810 18:51:19.922833 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:19.922835 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:19.922894 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:19.923033 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:51:19.923289 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.923891 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.925023 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:19.925254 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:19.925345 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:19.925620 (+ 275us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:51:19.926057 (+ 437us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:51:19.926149 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:19.926206 (+ 57us) server_negotiation.cc:299] Negotiation successful
0810 18:51:19.926254 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 18:51:19.932574 4657 raft_consensus.cc:1261] T 45015a60705843d2a86237efdd42014a P a5f555d7ab0d43248673885b404448ca [term 1 FOLLOWER]: Refusing update from remote peer 7ab2371396004cfc840d46f847dfd2f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:51:19.932595 4815 raft_consensus.cc:1261] T 45015a60705843d2a86237efdd42014a P 89fb620a5220480c9887a6a0d6827f19 [term 1 FOLLOWER]: Refusing update from remote peer 7ab2371396004cfc840d46f847dfd2f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:51:19.933326 4906 consensus_queue.cc:1035] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5f555d7ab0d43248673885b404448ca" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:51:19.933563 4898 consensus_queue.cc:1035] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89fb620a5220480c9887a6a0d6827f19" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:51:19.941788 4911 mvcc.cc:205] Tried to move back new op lower bound from 7317044756195713024 to 7317044755900379136. Current Snapshot: MvccSnapshot[applied={T|T < 7317044756195713024}]
I20260810 18:51:19.950948 4338 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57568:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:51:19.951090 4338 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57568:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:51:19.953644 4338 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162: Sending DeleteTablet for 3 replicas of tablet 45015a60705843d2a86237efdd42014a
I20260810 18:51:19.954475 4634 tablet_service.cc:1514] Processing DeleteTablet for tablet 45015a60705843d2a86237efdd42014a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:51:19 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:52602
I20260810 18:51:19.954569 4792 tablet_service.cc:1514] Processing DeleteTablet for tablet 45015a60705843d2a86237efdd42014a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:51:19 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:54032
I20260810 18:51:19.954569 4482 tablet_service.cc:1514] Processing DeleteTablet for tablet 45015a60705843d2a86237efdd42014a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:51:19 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:34930
I20260810 18:51:19.955067 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 4407
I20260810 18:51:19.955158 4923 tablet_replica.cc:330] T 45015a60705843d2a86237efdd42014a P 89fb620a5220480c9887a6a0d6827f19: stopping tablet replica
I20260810 18:51:19.957500 4923 raft_consensus.cc:2229] T 45015a60705843d2a86237efdd42014a P 89fb620a5220480c9887a6a0d6827f19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:51:19.957837 4923 raft_consensus.cc:2258] T 45015a60705843d2a86237efdd42014a P 89fb620a5220480c9887a6a0d6827f19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:51:19.958185 4921 tablet_replica.cc:330] T 45015a60705843d2a86237efdd42014a P a5f555d7ab0d43248673885b404448ca: stopping tablet replica
I20260810 18:51:19.958488 4921 raft_consensus.cc:2229] T 45015a60705843d2a86237efdd42014a P a5f555d7ab0d43248673885b404448ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:51:19.958781 4921 raft_consensus.cc:2258] T 45015a60705843d2a86237efdd42014a P a5f555d7ab0d43248673885b404448ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:51:19.961706 4923 ts_tablet_manager.cc:1905] T 45015a60705843d2a86237efdd42014a P 89fb620a5220480c9887a6a0d6827f19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:51:19.962823 4921 ts_tablet_manager.cc:1905] T 45015a60705843d2a86237efdd42014a P a5f555d7ab0d43248673885b404448ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:51:19.964591 4923 ts_tablet_manager.cc:1918] T 45015a60705843d2a86237efdd42014a P 89fb620a5220480c9887a6a0d6827f19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:51:19.964725 4923 log.cc:1196] T 45015a60705843d2a86237efdd42014a P 89fb620a5220480c9887a6a0d6827f19: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/wal/wals/45015a60705843d2a86237efdd42014a
I20260810 18:51:19.965166 4923 ts_tablet_manager.cc:1939] T 45015a60705843d2a86237efdd42014a P 89fb620a5220480c9887a6a0d6827f19: Deleting consensus metadata
I20260810 18:51:19.966290 4323 catalog_manager.cc:4841] TS 89fb620a5220480c9887a6a0d6827f19 (127.18.168.131:43805): tablet 45015a60705843d2a86237efdd42014a (table test-table [id=2295e10906f944d8a4589e525a0c6444]) successfully deleted
I20260810 18:51:19.966521 4921 ts_tablet_manager.cc:1918] T 45015a60705843d2a86237efdd42014a P a5f555d7ab0d43248673885b404448ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:51:19.966650 4921 log.cc:1196] T 45015a60705843d2a86237efdd42014a P a5f555d7ab0d43248673885b404448ca: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/wal/wals/45015a60705843d2a86237efdd42014a
I20260810 18:51:19.966991 4921 ts_tablet_manager.cc:1939] T 45015a60705843d2a86237efdd42014a P a5f555d7ab0d43248673885b404448ca: Deleting consensus metadata
I20260810 18:51:19.968191 4324 catalog_manager.cc:4841] TS a5f555d7ab0d43248673885b404448ca (127.18.168.130:39705): tablet 45015a60705843d2a86237efdd42014a (table test-table [id=2295e10906f944d8a4589e525a0c6444]) successfully deleted
W20260810 18:51:19.970741 4323 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.168.129:39577 (error 108)
I20260810 18:51:19.970975 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 4558
I20260810 18:51:19.973289 4883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:19.972194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:39577 (local address 127.0.0.1:34958)
0810 18:51:19.973111 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:19.973206 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.129:39577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":834}
W20260810 18:51:19.973443 4323 catalog_manager.cc:4584] TS 7ab2371396004cfc840d46f847dfd2f3 (127.18.168.129:39577): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 45015a60705843d2a86237efdd42014a: Network error: Client connection negotiation failed: client connection to 127.18.168.129:39577: connect: Connection refused (error 111)
I20260810 18:51:19.978571 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 4715
I20260810 18:51:19.985929 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 4305
I20260810 18:51:39.994987 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:41883
--webserver_interface=127.18.168.190
--webserver_port=46135
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:33555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:41883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:51:40.114113 4925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:51:40.114363 4925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:51:40.114403 4925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:51:40.118530 4925 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 18:51:40.118578 4925 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 18:51:40.118600 4925 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 18:51:40.118614 4925 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:51:40.118672 4925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:51:40.118705 4925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:51:40.118739 4925 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:51:40.118793 4925 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:51:40.124094 4925 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:33555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:41883
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:41883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=46135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4925
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:51:40.125195 4925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:51:40.126271 4925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:51:40.141281 4932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:40.141265 4935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:40.141283 4931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:40.141283 4925 server_base.cc:1034] running on GCE node
I20260810 18:51:40.142180 4925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:51:40.143441 4925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:51:40.144649 4925 hybrid_clock.cc:648] HybridClock initialized: now 1786387900144638 us; error 30 us; skew 500 ppm
Aug 10 18:51:40 dist-test-slave-cbsf krb5kdc[4287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387900, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:40.151245 4925 init.cc:377] Logged in from keytab as kudu/127.18.168.190@KRBTEST.COM (short username kudu)
I20260810 18:51:40.152235 4925 webserver.cc:460] Webserver started at http://127.18.168.190:46135/ using document root <none> and password file <none>
I20260810 18:51:40.152912 4925 fs_manager.cc:359] Metadata directory not provided
I20260810 18:51:40.153031 4925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:51:40.157027 4925 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 18:51:40.159305 4942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:40.160517 4925 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:51:40.160667 4925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "82542fe70878437fb5293cf40bd85162"
format_stamp: "Formatted at 2026-08-10 18:51:18 on dist-test-slave-cbsf"
server_key: "fccb38ac37c35ca792f1674f0133c448"
server_key_iv: "8f470552633b7aaceb5fb7f80da274da"
server_key_version: "encryptionkey@0"
I20260810 18:51:40.161406 4925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:51:40.184880 4925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:51:40.187832 4925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:51:40.188050 4925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:51:40.195559 4925 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:41883
I20260810 18:51:40.195878 5002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:41883 every 8 connection(s)
I20260810 18:51:40.196664 4925 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:51:40.201197 5003 sys_catalog.cc:263] Verifying existing consensus state
I20260810 18:51:40.202916 5003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162: Bootstrap starting.
I20260810 18:51:40.204051 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 4925
I20260810 18:51:40.204505 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.129:39577
--local_ip_for_outbound_sockets=127.18.168.129
--tserver_master_addrs=127.18.168.190:41883
--webserver_port=40433
--webserver_interface=127.18.168.129
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:33555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:51:40.221705 5003 log.cc:824] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162: Log is configured to *not* fsync() on all Append() calls
I20260810 18:51:40.230675 5003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:51:40.231050 5003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162: Bootstrap complete.
I20260810 18:51:40.235610 5003 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82542fe70878437fb5293cf40bd85162" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41883 } }
I20260810 18:51:40.236037 5003 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82542fe70878437fb5293cf40bd85162, State: Initialized, Role: FOLLOWER
I20260810 18:51:40.236517 5003 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82542fe70878437fb5293cf40bd85162" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41883 } }
I20260810 18:51:40.236672 5003 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:51:40.236788 5003 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:51:40.236904 5003 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [term 1 FOLLOWER]: Advancing to term 2
I20260810 18:51:40.238792 5003 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82542fe70878437fb5293cf40bd85162" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41883 } }
I20260810 18:51:40.239125 5003 leader_election.cc:302] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82542fe70878437fb5293cf40bd85162; no voters:
I20260810 18:51:40.239476 5003 leader_election.cc:288] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 18:51:40.239565 5007 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 18:51:40.239828 5007 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [term 2 LEADER]: Becoming Leader. State: Replica: 82542fe70878437fb5293cf40bd85162, State: Running, Role: LEADER
I20260810 18:51:40.240191 5007 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82542fe70878437fb5293cf40bd85162" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41883 } }
I20260810 18:51:40.240821 5003 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:51:40.241541 5009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "82542fe70878437fb5293cf40bd85162" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82542fe70878437fb5293cf40bd85162" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41883 } } }
I20260810 18:51:40.241667 5009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [sys.catalog]: This master's current role is: LEADER
I20260810 18:51:40.242400 5012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82542fe70878437fb5293cf40bd85162. Latest consensus state: current_term: 2 leader_uuid: "82542fe70878437fb5293cf40bd85162" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82542fe70878437fb5293cf40bd85162" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41883 } } }
I20260810 18:51:40.242482 5012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [sys.catalog]: This master's current role is: LEADER
I20260810 18:51:40.242878 5018 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:51:40.246992 5018 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:51:40.247535 5018 catalog_manager.cc:1259] Loaded cluster ID: 00f351aa82bc47d084c4fd8596cfdfe6
I20260810 18:51:40.247619 5018 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:51:40.249516 5018 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:51:40.250120 5018 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162: Loaded TSK: 0
I20260810 18:51:40.258488 5018 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162: Generated new TSK 1
I20260810 18:51:40.258744 5018 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 18:51:40.333341 5005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:51:40.333636 5005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:51:40.333724 5005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:51:40.338405 5005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:51:40.338501 5005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:51:40.338596 5005 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.129
I20260810 18:51:40.344779 5005 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:33555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.129:39577
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.168.129
--webserver_port=40433
--tserver_master_addrs=127.18.168.190:41883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5005
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.129
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:51:40.345844 5005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:51:40.346890 5005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:51:40.358103 5039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:40.358088 5036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:40.358094 5037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:40.358830 5005 server_base.cc:1034] running on GCE node
I20260810 18:51:40.359272 5005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:51:40.360009 5005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:51:40.361207 5005 hybrid_clock.cc:648] HybridClock initialized: now 1786387900361190 us; error 32 us; skew 500 ppm
Aug 10 18:51:40 dist-test-slave-cbsf krb5kdc[4287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387900, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:40.364428 5005 init.cc:377] Logged in from keytab as kudu/127.18.168.129@KRBTEST.COM (short username kudu)
I20260810 18:51:40.365617 5005 webserver.cc:460] Webserver started at http://127.18.168.129:40433/ using document root <none> and password file <none>
I20260810 18:51:40.366289 5005 fs_manager.cc:359] Metadata directory not provided
I20260810 18:51:40.366371 5005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:51:40.370774 5005 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260810 18:51:40.373307 5052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:40.374825 5005 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:51:40.374986 5005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/wal
uuid: "7ab2371396004cfc840d46f847dfd2f3"
format_stamp: "Formatted at 2026-08-10 18:51:19 on dist-test-slave-cbsf"
server_key: "3e3eab977583fb2a067fc976e18dc135"
server_key_iv: "38133292ddfdaec26ef612aa5eaef318"
server_key_version: "encryptionkey@0"
I20260810 18:51:40.375845 5005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:51:40.394974 5005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:51:40.399035 5005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:51:40.399307 5005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:51:40.400203 5005 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:51:40.401859 5059 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 18:51:40.407485 5005 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 18:51:40.407552 5005 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260810 18:51:40.407634 5005 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 18:51:40.409569 5005 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 18:51:40.409617 5005 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260810 18:51:40.409926 5059 tablet_bootstrap.cc:492] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3: Bootstrap starting.
I20260810 18:51:40.421452 5005 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.129:39577
I20260810 18:51:40.421482 5182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.129:39577 every 8 connection(s)
I20260810 18:51:40.422518 5005 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
I20260810 18:51:40.431563 5059 log.cc:824] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3: Log is configured to *not* fsync() on all Append() calls
I20260810 18:51:40.432787 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 5005
I20260810 18:51:40.433374 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.130:39705
--local_ip_for_outbound_sockets=127.18.168.130
--tserver_master_addrs=127.18.168.190:41883
--webserver_port=44477
--webserver_interface=127.18.168.130
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:33555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 18:51:40 dist-test-slave-cbsf krb5kdc[4287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387900, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:51:40.441053 5185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:40.424878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.129:40223)
0810 18:51:40.425419 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:40.425479 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:40.426423 (+ 944us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:40.427766 (+ 1343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:40.427773 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:40.428150 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:51:40.428626 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:40.428640 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:40.430153 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:40.430156 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:40.431067 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:40.431077 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:40.431947 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:40.431952 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:40.432215 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:40.433096 (+ 881us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:51:40.433145 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:51:40.435206 (+ 2061us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:51:40.437641 (+ 2435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:40.437647 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:40.437660 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:40.437955 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:40.438336 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:40.438341 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:40.438343 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:40.438535 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:40.439271 (+ 736us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:51:40.439277 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:51:40.439645 (+ 368us) client_negotiation.cc:769] Sending connection context
0810 18:51:40.439847 (+ 202us) client_negotiation.cc:240] Negotiation successful
0810 18:51:40.440194 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":160,"threads_started":1}
I20260810 18:51:40.441157 5059 tablet_bootstrap.cc:492] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:51:40.441669 5059 tablet_bootstrap.cc:492] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3: Bootstrap complete.
I20260810 18:51:40.442865 5186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:40.424942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:40223 (local address 127.18.168.190:41883)
0810 18:51:40.425494 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:40.425513 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:40.425588 (+ 75us) server_negotiation.cc:407] Connection header received
0810 18:51:40.426913 (+ 1325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:40.426929 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:40.427342 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:40.427723 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:40.428877 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:40.429957 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:40.431206 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:40.431768 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:40.431972 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:40.435498 (+ 3526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:40.435503 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:40.435517 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:40.435546 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:40.437494 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:40.438098 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:40.438103 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:40.438108 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:40.438229 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:40.438659 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:40.438663 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:40.438664 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:40.439109 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:40.439301 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:40.441705 (+ 2404us) server_negotiation.cc:299] Negotiation successful
0810 18:51:40.442026 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":148,"threads_started":1}
I20260810 18:51:40.443502 5059 ts_tablet_manager.cc:1397] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3: Time spent bootstrapping tablet: real 0.034s user 0.022s sys 0.009s
I20260810 18:51:40.445191 5183 heartbeater.cc:347] Connected to a master server at 127.18.168.190:41883
I20260810 18:51:40.445448 5183 heartbeater.cc:464] Registering TS with master...
I20260810 18:51:40.445964 5183 heartbeater.cc:511] Master 127.18.168.190:41883 requested a full tablet report, sending...
I20260810 18:51:40.447391 5059 raft_consensus.cc:348] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab2371396004cfc840d46f847dfd2f3" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39577 } } peers { permanent_uuid: "a5f555d7ab0d43248673885b404448ca" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39705 } } peers { permanent_uuid: "89fb620a5220480c9887a6a0d6827f19" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43805 } }
I20260810 18:51:40.447914 5059 raft_consensus.cc:729] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ab2371396004cfc840d46f847dfd2f3, State: Initialized, Role: FOLLOWER
I20260810 18:51:40.448403 5059 consensus_queue.cc:260] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab2371396004cfc840d46f847dfd2f3" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39577 } } peers { permanent_uuid: "a5f555d7ab0d43248673885b404448ca" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39705 } } peers { permanent_uuid: "89fb620a5220480c9887a6a0d6827f19" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43805 } }
I20260810 18:51:40.448495 4960 ts_manager.cc:194] Registered new tserver with Master: 7ab2371396004cfc840d46f847dfd2f3 (127.18.168.129:39577)
I20260810 18:51:40.449431 5059 ts_tablet_manager.cc:1428] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260810 18:51:40.452524 4960 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.129@KRBTEST.COM'} at 127.18.168.129:40223
I20260810 18:51:40.455118 5183 heartbeater.cc:503] Master 127.18.168.190:41883 was elected leader, sending a full tablet report...
Aug 10 18:51:40 dist-test-slave-cbsf krb5kdc[4287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387900, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for kudu/127.18.168.129@KRBTEST.COM
I20260810 18:51:40.462994 5191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:40.451620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:39577 (local address 127.0.0.1:55282)
0810 18:51:40.452055 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:40.452103 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:40.452226 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:40.452895 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:40.452902 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:40.453360 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:51:40.453509 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:40.453518 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:40.455068 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:40.455072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:40.456003 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:40.456013 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:40.456822 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:40.456826 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:40.456941 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:40.457502 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:51:40.457517 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:51:40.459158 (+ 1641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:51:40.461385 (+ 2227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:40.461393 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:40.461397 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:40.461623 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:40.461949 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:40.461951 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:40.461952 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:40.462030 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:40.462525 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:51:40.462529 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:51:40.462621 (+ 92us) client_negotiation.cc:769] Sending connection context
0810 18:51:40.462741 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 18:51:40.462877 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":111,"threads_started":1}
I20260810 18:51:40.463178 5192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:40.451686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55282 (local address 127.18.168.129:39577)
0810 18:51:40.452416 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:40.452429 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:40.452450 (+ 21us) server_negotiation.cc:407] Connection header received
0810 18:51:40.452515 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:40.452520 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:40.452733 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:40.452863 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:40.453649 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:40.454925 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:40.456130 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:40.456510 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:40.456826 (+ 316us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:40.459368 (+ 2542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:40.459380 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:40.459395 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:40.459422 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:40.461265 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:40.461749 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:40.461754 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:40.461756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:40.461851 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:40.462120 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:40.462123 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:40.462125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:40.462398 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:40.462532 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:40.462842 (+ 310us) server_negotiation.cc:299] Negotiation successful
0810 18:51:40.462998 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":106,"threads_started":1}
I20260810 18:51:40.464051 5106 tablet_service.cc:1514] Processing DeleteTablet for tablet 45015a60705843d2a86237efdd42014a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:51:19 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:55282
I20260810 18:51:40.465049 5193 tablet_replica.cc:330] stopping tablet replica
I20260810 18:51:40.465369 5193 raft_consensus.cc:2229] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:51:40.465644 5193 raft_consensus.cc:2258] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:51:40.466761 5193 ts_tablet_manager.cc:1905] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:51:40.471357 5193 ts_tablet_manager.cc:1918] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:51:40.471458 5193 log.cc:1196] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/wal/wals/45015a60705843d2a86237efdd42014a
I20260810 18:51:40.471750 5193 ts_tablet_manager.cc:1939] T 45015a60705843d2a86237efdd42014a P 7ab2371396004cfc840d46f847dfd2f3: Deleting consensus metadata
I20260810 18:51:40.473701 4945 catalog_manager.cc:4841] TS 7ab2371396004cfc840d46f847dfd2f3 (127.18.168.129:39577): tablet 45015a60705843d2a86237efdd42014a (table test-table [id=2295e10906f944d8a4589e525a0c6444]) successfully deleted
W20260810 18:51:40.575199 5188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:51:40.575435 5188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:51:40.575475 5188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:51:40.579620 5188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:51:40.579672 5188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:51:40.579777 5188 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.130
I20260810 18:51:40.585704 5188 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:33555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.130:39705
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.168.130
--webserver_port=44477
--tserver_master_addrs=127.18.168.190:41883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.130
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:51:40.586802 5188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:51:40.587864 5188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:51:40.597996 5202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:40.598114 5188 server_base.cc:1034] running on GCE node
W20260810 18:51:40.598032 5204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:40.598006 5201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:40.598664 5188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:51:40.599352 5188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:51:40.600543 5188 hybrid_clock.cc:648] HybridClock initialized: now 1786387900600504 us; error 51 us; skew 500 ppm
Aug 10 18:51:40 dist-test-slave-cbsf krb5kdc[4287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387900, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:40.603340 5188 init.cc:377] Logged in from keytab as kudu/127.18.168.130@KRBTEST.COM (short username kudu)
I20260810 18:51:40.604167 5188 webserver.cc:460] Webserver started at http://127.18.168.130:44477/ using document root <none> and password file <none>
I20260810 18:51:40.604759 5188 fs_manager.cc:359] Metadata directory not provided
I20260810 18:51:40.604856 5188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:51:40.608774 5188 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 18:51:40.610824 5211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:40.612206 5188 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 18:51:40.612330 5188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/wal
uuid: "a5f555d7ab0d43248673885b404448ca"
format_stamp: "Formatted at 2026-08-10 18:51:19 on dist-test-slave-cbsf"
server_key: "04b6d50707eb5d13c4bae123288f2742"
server_key_iv: "4d89f223ce06e066b64872d6da3489e2"
server_key_version: "encryptionkey@0"
I20260810 18:51:40.612958 5188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:51:40.627451 5188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:51:40.630630 5188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:51:40.630868 5188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:51:40.631516 5188 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:51:40.632541 5188 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:51:40.632591 5188 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:40.632663 5188 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:51:40.632761 5188 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260810 18:51:40.642445 5188 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.130:39705
I20260810 18:51:40.642490 5336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.130:39705 every 8 connection(s)
I20260810 18:51:40.643535 5188 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
I20260810 18:51:40.652558 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 5188
Aug 10 18:51:40 dist-test-slave-cbsf krb5kdc[4287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387900, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:51:40.653029 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.131:43805
--local_ip_for_outbound_sockets=127.18.168.131
--tserver_master_addrs=127.18.168.190:41883
--webserver_port=42121
--webserver_interface=127.18.168.131
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:33555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:51:40.657814 5186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:40.645407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:46483 (local address 127.18.168.190:41883)
0810 18:51:40.645560 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:40.645564 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:40.646411 (+ 847us) server_negotiation.cc:407] Connection header received
0810 18:51:40.647707 (+ 1296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:40.647710 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:40.647761 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:40.647837 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:40.649061 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:40.649593 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:40.650707 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:40.650934 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:40.651010 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:40.653697 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:40.653704 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:40.653710 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:40.653746 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:40.655383 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:40.656053 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:40.656057 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:40.656058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:40.656140 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:40.656570 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:40.656573 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:40.656574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:40.656821 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:40.656901 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:40.657561 (+ 660us) server_negotiation.cc:299] Negotiation successful
0810 18:51:40.657688 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 18:51:40.659091 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:40.645809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.130:46483)
0810 18:51:40.646257 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:40.646300 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:40.647500 (+ 1200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:40.648004 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:40.648010 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:40.648365 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:51:40.648930 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:40.648946 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:40.649703 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:40.649705 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:40.650599 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:40.650608 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:40.651063 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:40.651065 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:40.651212 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:40.651836 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:51:40.651852 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:51:40.653513 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:51:40.655508 (+ 1995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:40.655516 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:40.655534 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:40.655937 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:40.656243 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:40.656248 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:40.656253 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:40.656468 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:40.656934 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:51:40.656944 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:51:40.657299 (+ 355us) client_negotiation.cc:769] Sending connection context
0810 18:51:40.657525 (+ 226us) client_negotiation.cc:240] Negotiation successful
0810 18:51:40.657907 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":111,"threads_started":1}
I20260810 18:51:40.660382 5337 heartbeater.cc:347] Connected to a master server at 127.18.168.190:41883
I20260810 18:51:40.660637 5337 heartbeater.cc:464] Registering TS with master...
I20260810 18:51:40.661175 5337 heartbeater.cc:511] Master 127.18.168.190:41883 requested a full tablet report, sending...
I20260810 18:51:40.662312 4960 ts_manager.cc:194] Registered new tserver with Master: a5f555d7ab0d43248673885b404448ca (127.18.168.130:39705)
I20260810 18:51:40.662887 4960 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.130@KRBTEST.COM'} at 127.18.168.130:46483
W20260810 18:51:40.774463 5342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:51:40.774757 5342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:51:40.774852 5342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:51:40.780241 5342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:51:40.780323 5342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:51:40.780408 5342 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.131
I20260810 18:51:40.785880 5342 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:33555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.131:43805
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.168.131
--webserver_port=42121
--tserver_master_addrs=127.18.168.190:41883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.131
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:51:40.786947 5342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:51:40.787993 5342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:51:40.798062 5351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:40.798048 5349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:40.798128 5348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:40.798344 5342 server_base.cc:1034] running on GCE node
I20260810 18:51:40.798736 5342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:51:40.799361 5342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:51:40.800568 5342 hybrid_clock.cc:648] HybridClock initialized: now 1786387900800542 us; error 45 us; skew 500 ppm
Aug 10 18:51:40 dist-test-slave-cbsf krb5kdc[4287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387900, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:40.803354 5342 init.cc:377] Logged in from keytab as kudu/127.18.168.131@KRBTEST.COM (short username kudu)
I20260810 18:51:40.804222 5342 webserver.cc:460] Webserver started at http://127.18.168.131:42121/ using document root <none> and password file <none>
I20260810 18:51:40.804889 5342 fs_manager.cc:359] Metadata directory not provided
I20260810 18:51:40.805003 5342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:51:40.808835 5342 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260810 18:51:40.810956 5360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:40.812199 5342 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:51:40.812338 5342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/wal
uuid: "89fb620a5220480c9887a6a0d6827f19"
format_stamp: "Formatted at 2026-08-10 18:51:19 on dist-test-slave-cbsf"
server_key: "8e2ec631dcc193d64d1d723ed5621860"
server_key_iv: "13c7ef17bdfe814e892f79c8a14fc294"
server_key_version: "encryptionkey@0"
I20260810 18:51:40.812978 5342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:51:40.829761 5342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:51:40.832623 5342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:51:40.832850 5342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:51:40.833534 5342 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:51:40.834409 5342 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:51:40.834481 5342 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:40.834553 5342 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:51:40.834607 5342 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:40.844225 5342 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.131:43805
I20260810 18:51:40.844317 5493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.131:43805 every 8 connection(s)
I20260810 18:51:40.845289 5342 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
I20260810 18:51:40.852209 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 5342
Aug 10 18:51:40 dist-test-slave-cbsf krb5kdc[4287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387900, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:51:40.859630 5186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:40.847193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:59681 (local address 127.18.168.190:41883)
0810 18:51:40.847348 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:40.847352 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:40.848196 (+ 844us) server_negotiation.cc:407] Connection header received
0810 18:51:40.849561 (+ 1365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:40.849564 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:40.849615 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:40.849693 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:40.850846 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:40.851364 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:40.852609 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:40.853007 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:40.853147 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:40.855617 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:40.855622 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:40.855624 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:40.855651 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:40.857252 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:40.857793 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:40.857796 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:40.857798 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:40.857887 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:40.858340 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:40.858342 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:40.858343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:40.858575 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:40.858682 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:40.859331 (+ 649us) server_negotiation.cc:299] Negotiation successful
0810 18:51:40.859430 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 18:51:40.860529 5496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:40.847566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.131:59681)
0810 18:51:40.848038 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:40.848081 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:40.849360 (+ 1279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:40.849862 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:40.849869 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:40.850240 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:51:40.850714 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:40.850727 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:40.851471 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:40.851473 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:40.852473 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:40.852484 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:40.853191 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:40.853196 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:40.853368 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:40.853987 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:51:40.854003 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:51:40.855441 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:51:40.857378 (+ 1937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:40.857384 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:40.857400 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:40.857679 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:40.858010 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:40.858013 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:40.858014 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:40.858227 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:40.858705 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:51:40.858715 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:51:40.859053 (+ 338us) client_negotiation.cc:769] Sending connection context
0810 18:51:40.859290 (+ 237us) client_negotiation.cc:240] Negotiation successful
0810 18:51:40.859641 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":114,"threads_started":1}
I20260810 18:51:40.861486 5498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:40.853003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34088 (local address 127.18.168.190:41883)
0810 18:51:40.853296 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:40.853301 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:40.853484 (+ 183us) server_negotiation.cc:407] Connection header received
0810 18:51:40.853646 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:40.853649 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:40.853703 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:40.853800 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:40.854551 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:40.855101 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:40.856007 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:40.856248 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:40.856355 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:40.857680 (+ 1325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:40.857683 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:40.857686 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:40.857711 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:40.859414 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:40.859918 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:40.859921 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:40.859925 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:40.860002 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:40.860290 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:40.860294 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:40.860295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:40.860459 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:40.860678 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:40.861178 (+ 500us) server_negotiation.cc:299] Negotiation successful
0810 18:51:40.861310 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":96,"threads_started":1}
I20260810 18:51:40.861913 5494 heartbeater.cc:347] Connected to a master server at 127.18.168.190:41883
I20260810 18:51:40.862185 5494 heartbeater.cc:464] Registering TS with master...
I20260810 18:51:40.862766 5494 heartbeater.cc:511] Master 127.18.168.190:41883 requested a full tablet report, sending...
I20260810 18:51:40.863840 4960 ts_manager.cc:194] Registered new tserver with Master: 89fb620a5220480c9887a6a0d6827f19 (127.18.168.131:43805)
I20260810 18:51:40.864391 4960 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.131@KRBTEST.COM'} at 127.18.168.131:59681
I20260810 18:51:40.874925 19106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:51:40.879002 5498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:40.875578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34092 (local address 127.18.168.190:41883)
0810 18:51:40.875723 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:40.875728 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:40.875962 (+ 234us) server_negotiation.cc:407] Connection header received
0810 18:51:40.876085 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:40.876088 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:40.876189 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:40.876278 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:51:40.876501 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:40.877100 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:40.877946 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:40.878168 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:40.878245 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:40.878512 (+ 267us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:51:40.878658 (+ 146us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0810 18:51:40.878857 (+ 199us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34092: token expired
Metrics: {"server-negotiator.queue_time_us":51}
W20260810 18:51:40.879081 5498 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34092: token expired
W20260810 18:51:40.879504 4876 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.18.168.190:41883)
I20260810 18:51:40.886725 5498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:40.879894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34100 (local address 127.18.168.190:41883)
0810 18:51:40.880010 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:40.880013 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:40.880025 (+ 12us) server_negotiation.cc:407] Connection header received
0810 18:51:40.880157 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:40.880159 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:40.880194 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:40.880257 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:40.881025 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:40.881492 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:40.882233 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:40.882425 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:40.882496 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:40.883784 (+ 1288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:40.883787 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:40.883789 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:40.883811 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:40.885323 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:40.885723 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:40.885725 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:40.885726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:40.885799 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:40.886032 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:40.886034 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:40.886035 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:40.886202 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:40.886300 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:40.886526 (+ 226us) server_negotiation.cc:299] Negotiation successful
0810 18:51:40.886616 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 18:51:40.889128 4960 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34100:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:51:40.892411 4960 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:51:40.900669 5106 tablet_service.cc:1467] Processing CreateTablet for tablet 23af3c75848d4077bab974c59df468fb (DEFAULT_TABLE table=test-table [id=8546e39ff25e412188347d5882fe59d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:51:40.901448 5106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23af3c75848d4077bab974c59df468fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:51:40.904361 5059 tablet_bootstrap.cc:492] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3: Bootstrap starting.
I20260810 18:51:40.905362 5059 tablet_bootstrap.cc:654] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3: Neither blocks nor log segments found. Creating new log.
I20260810 18:51:40.905758 5191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:40.899385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.131:43805 (local address 127.0.0.1:37692)
0810 18:51:40.899520 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:40.899537 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:40.899619 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:40.900901 (+ 1282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:40.900904 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:40.900924 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:51:40.901037 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:40.901043 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:40.902785 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:40.902788 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:40.904067 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:40.904080 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:40.905272 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:40.905276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:40.905381 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:40.905433 (+ 52us) client_negotiation.cc:769] Sending connection context
0810 18:51:40.905507 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 18:51:40.905591 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":24}
I20260810 18:51:40.906134 5506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:40.899843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37692 (local address 127.18.168.131:43805)
0810 18:51:40.900335 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:40.900348 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:40.900373 (+ 25us) server_negotiation.cc:407] Connection header received
0810 18:51:40.900456 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:40.900462 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:40.900719 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:40.900913 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:51:40.901309 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:40.902645 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:40.904202 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:40.905132 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:40.905288 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:40.905570 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:40.905655 (+ 85us) server_negotiation.cc:299] Negotiation successful
0810 18:51:40.905771 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":159,"threads_started":1}
I20260810 18:51:40.906424 5504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:40.899688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:39705 (local address 127.0.0.1:37784)
0810 18:51:40.900066 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:40.900081 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:40.900166 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:40.901606 (+ 1440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:40.901610 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:40.901630 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:51:40.901720 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:40.901726 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:40.903013 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:40.903017 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:40.903935 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:40.903947 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:40.905937 (+ 1990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:40.905941 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:40.906077 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:40.906096 (+ 19us) client_negotiation.cc:769] Sending connection context
0810 18:51:40.906187 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 18:51:40.906257 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":37,"thread_start_us":164,"threads_started":1}
I20260810 18:51:40.906838 5505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:40.899873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37784 (local address 127.18.168.130:39705)
0810 18:51:40.900933 (+ 1060us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:40.900946 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:40.900967 (+ 21us) server_negotiation.cc:407] Connection header received
0810 18:51:40.901040 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:40.901045 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:40.901258 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:40.901415 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:51:40.901859 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:40.902902 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:40.904049 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:40.905611 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:40.905738 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:40.905867 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:40.906600 (+ 733us) server_negotiation.cc:299] Negotiation successful
0810 18:51:40.906715 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":937,"thread_start_us":116,"threads_started":1}
I20260810 18:51:40.906976 5059 tablet_bootstrap.cc:492] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3: No bootstrap required, opened a new log
I20260810 18:51:40.907080 5059 ts_tablet_manager.cc:1397] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 18:51:40.907549 5059 raft_consensus.cc:348] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fb620a5220480c9887a6a0d6827f19" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43805 } } peers { permanent_uuid: "a5f555d7ab0d43248673885b404448ca" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39705 } } peers { permanent_uuid: "7ab2371396004cfc840d46f847dfd2f3" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39577 } }
I20260810 18:51:40.907706 5059 raft_consensus.cc:374] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:51:40.907754 5059 raft_consensus.cc:729] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ab2371396004cfc840d46f847dfd2f3, State: Initialized, Role: FOLLOWER
I20260810 18:51:40.907918 5059 consensus_queue.cc:260] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fb620a5220480c9887a6a0d6827f19" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43805 } } peers { permanent_uuid: "a5f555d7ab0d43248673885b404448ca" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39705 } } peers { permanent_uuid: "7ab2371396004cfc840d46f847dfd2f3" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39577 } }
I20260810 18:51:40.908280 5059 ts_tablet_manager.cc:1428] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 18:51:40.909499 5419 tablet_service.cc:1467] Processing CreateTablet for tablet 23af3c75848d4077bab974c59df468fb (DEFAULT_TABLE table=test-table [id=8546e39ff25e412188347d5882fe59d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:51:40.910681 5419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23af3c75848d4077bab974c59df468fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:51:40.911564 5265 tablet_service.cc:1467] Processing CreateTablet for tablet 23af3c75848d4077bab974c59df468fb (DEFAULT_TABLE table=test-table [id=8546e39ff25e412188347d5882fe59d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:51:40.912624 5265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23af3c75848d4077bab974c59df468fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:51:40.918319 5509 tablet_bootstrap.cc:492] T 23af3c75848d4077bab974c59df468fb P 89fb620a5220480c9887a6a0d6827f19: Bootstrap starting.
I20260810 18:51:40.919236 5510 tablet_bootstrap.cc:492] T 23af3c75848d4077bab974c59df468fb P a5f555d7ab0d43248673885b404448ca: Bootstrap starting.
I20260810 18:51:40.920467 5509 tablet_bootstrap.cc:654] T 23af3c75848d4077bab974c59df468fb P 89fb620a5220480c9887a6a0d6827f19: Neither blocks nor log segments found. Creating new log.
I20260810 18:51:40.921531 5510 tablet_bootstrap.cc:654] T 23af3c75848d4077bab974c59df468fb P a5f555d7ab0d43248673885b404448ca: Neither blocks nor log segments found. Creating new log.
I20260810 18:51:40.921697 5509 log.cc:824] T 23af3c75848d4077bab974c59df468fb P 89fb620a5220480c9887a6a0d6827f19: Log is configured to *not* fsync() on all Append() calls
I20260810 18:51:40.922295 5510 log.cc:824] T 23af3c75848d4077bab974c59df468fb P a5f555d7ab0d43248673885b404448ca: Log is configured to *not* fsync() on all Append() calls
W20260810 18:51:40.923791 5184 tablet.cc:2369] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:51:40.923874 5509 tablet_bootstrap.cc:492] T 23af3c75848d4077bab974c59df468fb P 89fb620a5220480c9887a6a0d6827f19: No bootstrap required, opened a new log
I20260810 18:51:40.924072 5509 ts_tablet_manager.cc:1397] T 23af3c75848d4077bab974c59df468fb P 89fb620a5220480c9887a6a0d6827f19: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260810 18:51:40.924383 5510 tablet_bootstrap.cc:492] T 23af3c75848d4077bab974c59df468fb P a5f555d7ab0d43248673885b404448ca: No bootstrap required, opened a new log
I20260810 18:51:40.924572 5510 ts_tablet_manager.cc:1397] T 23af3c75848d4077bab974c59df468fb P a5f555d7ab0d43248673885b404448ca: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 18:51:40.928184 5509 raft_consensus.cc:348] T 23af3c75848d4077bab974c59df468fb P 89fb620a5220480c9887a6a0d6827f19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fb620a5220480c9887a6a0d6827f19" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43805 } } peers { permanent_uuid: "a5f555d7ab0d43248673885b404448ca" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39705 } } peers { permanent_uuid: "7ab2371396004cfc840d46f847dfd2f3" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39577 } }
I20260810 18:51:40.928488 5509 raft_consensus.cc:374] T 23af3c75848d4077bab974c59df468fb P 89fb620a5220480c9887a6a0d6827f19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:51:40.928355 5510 raft_consensus.cc:348] T 23af3c75848d4077bab974c59df468fb P a5f555d7ab0d43248673885b404448ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fb620a5220480c9887a6a0d6827f19" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43805 } } peers { permanent_uuid: "a5f555d7ab0d43248673885b404448ca" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39705 } } peers { permanent_uuid: "7ab2371396004cfc840d46f847dfd2f3" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39577 } }
I20260810 18:51:40.928550 5509 raft_consensus.cc:729] T 23af3c75848d4077bab974c59df468fb P 89fb620a5220480c9887a6a0d6827f19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89fb620a5220480c9887a6a0d6827f19, State: Initialized, Role: FOLLOWER
I20260810 18:51:40.928597 5510 raft_consensus.cc:374] T 23af3c75848d4077bab974c59df468fb P a5f555d7ab0d43248673885b404448ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:51:40.928654 5510 raft_consensus.cc:729] T 23af3c75848d4077bab974c59df468fb P a5f555d7ab0d43248673885b404448ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5f555d7ab0d43248673885b404448ca, State: Initialized, Role: FOLLOWER
I20260810 18:51:40.929068 5509 consensus_queue.cc:260] T 23af3c75848d4077bab974c59df468fb P 89fb620a5220480c9887a6a0d6827f19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fb620a5220480c9887a6a0d6827f19" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43805 } } peers { permanent_uuid: "a5f555d7ab0d43248673885b404448ca" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39705 } } peers { permanent_uuid: "7ab2371396004cfc840d46f847dfd2f3" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39577 } }
I20260810 18:51:40.929068 5510 consensus_queue.cc:260] T 23af3c75848d4077bab974c59df468fb P a5f555d7ab0d43248673885b404448ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fb620a5220480c9887a6a0d6827f19" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43805 } } peers { permanent_uuid: "a5f555d7ab0d43248673885b404448ca" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39705 } } peers { permanent_uuid: "7ab2371396004cfc840d46f847dfd2f3" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39577 } }
I20260810 18:51:40.930109 5510 ts_tablet_manager.cc:1428] T 23af3c75848d4077bab974c59df468fb P a5f555d7ab0d43248673885b404448ca: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260810 18:51:40.930117 5509 ts_tablet_manager.cc:1428] T 23af3c75848d4077bab974c59df468fb P 89fb620a5220480c9887a6a0d6827f19: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260810 18:51:40.930296 5337 heartbeater.cc:503] Master 127.18.168.190:41883 was elected leader, sending a full tablet report...
I20260810 18:51:40.930356 5494 heartbeater.cc:503] Master 127.18.168.190:41883 was elected leader, sending a full tablet report...
W20260810 18:51:41.096463 5495 tablet.cc:2369] T 23af3c75848d4077bab974c59df468fb P 89fb620a5220480c9887a6a0d6827f19: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:51:41.144837 5338 tablet.cc:2369] T 23af3c75848d4077bab974c59df468fb P a5f555d7ab0d43248673885b404448ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:51:41.171924 5514 raft_consensus.cc:482] T 23af3c75848d4077bab974c59df468fb P 89fb620a5220480c9887a6a0d6827f19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:51:41.172197 5514 raft_consensus.cc:504] T 23af3c75848d4077bab974c59df468fb P 89fb620a5220480c9887a6a0d6827f19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fb620a5220480c9887a6a0d6827f19" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43805 } } peers { permanent_uuid: "a5f555d7ab0d43248673885b404448ca" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39705 } } peers { permanent_uuid: "7ab2371396004cfc840d46f847dfd2f3" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39577 } }
I20260810 18:51:41.173512 5514 leader_election.cc:288] T 23af3c75848d4077bab974c59df468fb P 89fb620a5220480c9887a6a0d6827f19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5f555d7ab0d43248673885b404448ca (127.18.168.130:39705), 7ab2371396004cfc840d46f847dfd2f3 (127.18.168.129:39577)
I20260810 18:51:41.177812 5505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:41.173945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:38335 (local address 127.18.168.130:39705)
0810 18:51:41.174096 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:41.174102 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:41.174400 (+ 298us) server_negotiation.cc:407] Connection header received
0810 18:51:41.174566 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:41.174587 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:41.174645 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:41.174725 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:51:41.175061 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:41.175547 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:41.176806 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:41.177224 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:41.177322 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:41.177404 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:41.177596 (+ 192us) server_negotiation.cc:299] Negotiation successful
0810 18:51:41.177684 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 18:51:41.177811 5519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:41.173820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:39705 (local address 127.18.168.131:38335)
0810 18:51:41.174295 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:41.174307 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:41.174440 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:41.174763 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:41.174765 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:41.174782 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:51:41.174932 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:41.174939 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:41.175704 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:41.175707 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:41.176631 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:41.176641 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:41.177360 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:41.177367 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:41.177463 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:41.177487 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 18:51:41.177565 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 18:51:41.177645 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"mutex_wait_us":171,"thread_start_us":135,"threads_started":1}
I20260810 18:51:41.177826 5496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:41.173820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:39577 (local address 127.18.168.131:54421)
0810 18:51:41.173954 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:41.173971 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:41.174095 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:41.174629 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:41.174633 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:41.174651 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:51:41.174821 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:41.174829 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:41.175653 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:41.175658 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:41.176627 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:41.176637 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:41.177422 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:41.177426 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:41.177527 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:41.177573 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 18:51:41.177660 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 18:51:41.177721 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":84}
I20260810 18:51:41.178328 5517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:41.173921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:54421 (local address 127.18.168.129:39577)
0810 18:51:41.174249 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:41.174253 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:41.174265 (+ 12us) server_negotiation.cc:407] Connection header received
0810 18:51:41.174359 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:41.174361 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:41.174430 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:41.174589 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:51:41.174950 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:41.175522 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:41.176783 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:41.177264 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:41.177412 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:41.177580 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:41.177688 (+ 108us) server_negotiation.cc:299] Negotiation successful
0810 18:51:41.177761 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":115,"threads_started":1}
I20260810 18:51:41.178632 5286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23af3c75848d4077bab974c59df468fb" candidate_uuid: "89fb620a5220480c9887a6a0d6827f19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5f555d7ab0d43248673885b404448ca" is_pre_election: true
I20260810 18:51:41.178884 5128 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23af3c75848d4077bab974c59df468fb" candidate_uuid: "89fb620a5220480c9887a6a0d6827f19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ab2371396004cfc840d46f847dfd2f3" is_pre_election: true
I20260810 18:51:41.179018 5286 raft_consensus.cc:2454] T 23af3c75848d4077bab974c59df468fb P a5f555d7ab0d43248673885b404448ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89fb620a5220480c9887a6a0d6827f19 in term 0.
I20260810 18:51:41.179176 5128 raft_consensus.cc:2454] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89fb620a5220480c9887a6a0d6827f19 in term 0.
I20260810 18:51:41.179522 5366 leader_election.cc:302] T 23af3c75848d4077bab974c59df468fb P 89fb620a5220480c9887a6a0d6827f19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89fb620a5220480c9887a6a0d6827f19, a5f555d7ab0d43248673885b404448ca; no voters:
I20260810 18:51:41.179834 5514 raft_consensus.cc:2781] T 23af3c75848d4077bab974c59df468fb P 89fb620a5220480c9887a6a0d6827f19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:51:41.179922 5514 raft_consensus.cc:482] T 23af3c75848d4077bab974c59df468fb P 89fb620a5220480c9887a6a0d6827f19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:51:41.179955 5514 raft_consensus.cc:3037] T 23af3c75848d4077bab974c59df468fb P 89fb620a5220480c9887a6a0d6827f19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:51:41.181265 5514 raft_consensus.cc:504] T 23af3c75848d4077bab974c59df468fb P 89fb620a5220480c9887a6a0d6827f19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fb620a5220480c9887a6a0d6827f19" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43805 } } peers { permanent_uuid: "a5f555d7ab0d43248673885b404448ca" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39705 } } peers { permanent_uuid: "7ab2371396004cfc840d46f847dfd2f3" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39577 } }
I20260810 18:51:41.181634 5514 leader_election.cc:288] T 23af3c75848d4077bab974c59df468fb P 89fb620a5220480c9887a6a0d6827f19 [CANDIDATE]: Term 1 election: Requested vote from peers a5f555d7ab0d43248673885b404448ca (127.18.168.130:39705), 7ab2371396004cfc840d46f847dfd2f3 (127.18.168.129:39577)
I20260810 18:51:41.181983 5286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23af3c75848d4077bab974c59df468fb" candidate_uuid: "89fb620a5220480c9887a6a0d6827f19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5f555d7ab0d43248673885b404448ca"
I20260810 18:51:41.182016 5128 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23af3c75848d4077bab974c59df468fb" candidate_uuid: "89fb620a5220480c9887a6a0d6827f19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ab2371396004cfc840d46f847dfd2f3"
I20260810 18:51:41.182101 5286 raft_consensus.cc:3037] T 23af3c75848d4077bab974c59df468fb P a5f555d7ab0d43248673885b404448ca [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:51:41.182161 5128 raft_consensus.cc:3037] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:51:41.183205 5286 raft_consensus.cc:2454] T 23af3c75848d4077bab974c59df468fb P a5f555d7ab0d43248673885b404448ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89fb620a5220480c9887a6a0d6827f19 in term 1.
I20260810 18:51:41.183207 5128 raft_consensus.cc:2454] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89fb620a5220480c9887a6a0d6827f19 in term 1.
I20260810 18:51:41.183547 5364 leader_election.cc:302] T 23af3c75848d4077bab974c59df468fb P 89fb620a5220480c9887a6a0d6827f19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 7ab2371396004cfc840d46f847dfd2f3, 89fb620a5220480c9887a6a0d6827f19, a5f555d7ab0d43248673885b404448ca; no voters:
I20260810 18:51:41.183771 5514 raft_consensus.cc:2781] T 23af3c75848d4077bab974c59df468fb P 89fb620a5220480c9887a6a0d6827f19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:51:41.183975 5514 raft_consensus.cc:686] T 23af3c75848d4077bab974c59df468fb P 89fb620a5220480c9887a6a0d6827f19 [term 1 LEADER]: Becoming Leader. State: Replica: 89fb620a5220480c9887a6a0d6827f19, State: Running, Role: LEADER
I20260810 18:51:41.184314 5514 consensus_queue.cc:237] T 23af3c75848d4077bab974c59df468fb P 89fb620a5220480c9887a6a0d6827f19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fb620a5220480c9887a6a0d6827f19" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43805 } } peers { permanent_uuid: "a5f555d7ab0d43248673885b404448ca" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39705 } } peers { permanent_uuid: "7ab2371396004cfc840d46f847dfd2f3" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39577 } }
I20260810 18:51:41.187566 4960 catalog_manager.cc:5495] T 23af3c75848d4077bab974c59df468fb P 89fb620a5220480c9887a6a0d6827f19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 89fb620a5220480c9887a6a0d6827f19 (127.18.168.131). New cstate: current_term: 1 leader_uuid: "89fb620a5220480c9887a6a0d6827f19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fb620a5220480c9887a6a0d6827f19" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a5f555d7ab0d43248673885b404448ca" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ab2371396004cfc840d46f847dfd2f3" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39577 } health_report { overall_health: UNKNOWN } } }
I20260810 18:51:41.199467 5506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:41.195839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37708 (local address 127.18.168.131:43805)
0810 18:51:41.195987 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:41.195992 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:41.196006 (+ 14us) server_negotiation.cc:407] Connection header received
0810 18:51:41.196191 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:41.196194 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:41.196249 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:41.196360 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:51:41.196624 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:41.197207 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:41.198037 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:41.198267 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:41.198380 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:41.198695 (+ 315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:51:41.199060 (+ 365us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:51:41.199195 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:41.199256 (+ 61us) server_negotiation.cc:299] Negotiation successful
0810 18:51:41.199308 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 18:51:41.205334 5128 raft_consensus.cc:1261] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [term 1 FOLLOWER]: Refusing update from remote peer 89fb620a5220480c9887a6a0d6827f19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:51:41.205394 5286 raft_consensus.cc:1261] T 23af3c75848d4077bab974c59df468fb P a5f555d7ab0d43248673885b404448ca [term 1 FOLLOWER]: Refusing update from remote peer 89fb620a5220480c9887a6a0d6827f19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:51:41.206066 5520 consensus_queue.cc:1035] T 23af3c75848d4077bab974c59df468fb P 89fb620a5220480c9887a6a0d6827f19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ab2371396004cfc840d46f847dfd2f3" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:51:41.206264 5514 consensus_queue.cc:1035] T 23af3c75848d4077bab974c59df468fb P 89fb620a5220480c9887a6a0d6827f19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5f555d7ab0d43248673885b404448ca" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:51:41.222815 4960 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34100:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:51:41.223022 4960 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34100:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:51:41.225405 4960 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162: Sending DeleteTablet for 3 replicas of tablet 23af3c75848d4077bab974c59df468fb
I20260810 18:51:41.226119 5419 tablet_service.cc:1514] Processing DeleteTablet for tablet 23af3c75848d4077bab974c59df468fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:51:41 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:37692
I20260810 18:51:41.226171 5106 tablet_service.cc:1514] Processing DeleteTablet for tablet 23af3c75848d4077bab974c59df468fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:51:41 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:55282
I20260810 18:51:41.226603 5533 tablet_replica.cc:330] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3: stopping tablet replica
I20260810 18:51:41.226759 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 5005
I20260810 18:51:41.226779 5533 raft_consensus.cc:2229] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:51:41.226930 5533 raft_consensus.cc:2258] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:51:41.232733 5265 tablet_service.cc:1514] Processing DeleteTablet for tablet 23af3c75848d4077bab974c59df468fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:51:41 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:37784
I20260810 18:51:41.233342 5535 tablet_replica.cc:330] T 23af3c75848d4077bab974c59df468fb P a5f555d7ab0d43248673885b404448ca: stopping tablet replica
I20260810 18:51:41.233733 5535 raft_consensus.cc:2229] T 23af3c75848d4077bab974c59df468fb P a5f555d7ab0d43248673885b404448ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:51:41.234114 5535 raft_consensus.cc:2258] T 23af3c75848d4077bab974c59df468fb P a5f555d7ab0d43248673885b404448ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:51:41.235931 5535 ts_tablet_manager.cc:1905] T 23af3c75848d4077bab974c59df468fb P a5f555d7ab0d43248673885b404448ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:51:41.236943 5534 tablet_replica.cc:330] T 23af3c75848d4077bab974c59df468fb P 89fb620a5220480c9887a6a0d6827f19: stopping tablet replica
I20260810 18:51:41.237319 5534 raft_consensus.cc:2229] T 23af3c75848d4077bab974c59df468fb P 89fb620a5220480c9887a6a0d6827f19 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:51:41.237795 5534 raft_consensus.cc:2258] T 23af3c75848d4077bab974c59df468fb P 89fb620a5220480c9887a6a0d6827f19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:51:41.239292 5534 ts_tablet_manager.cc:1905] T 23af3c75848d4077bab974c59df468fb P 89fb620a5220480c9887a6a0d6827f19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:51:41.239610 5535 ts_tablet_manager.cc:1918] T 23af3c75848d4077bab974c59df468fb P a5f555d7ab0d43248673885b404448ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:51:41.239727 5535 log.cc:1196] T 23af3c75848d4077bab974c59df468fb P a5f555d7ab0d43248673885b404448ca: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/wal/wals/23af3c75848d4077bab974c59df468fb
I20260810 18:51:41.240106 5535 ts_tablet_manager.cc:1939] T 23af3c75848d4077bab974c59df468fb P a5f555d7ab0d43248673885b404448ca: Deleting consensus metadata
W20260810 18:51:41.240513 4945 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.168.129:39577 (error 108)
I20260810 18:51:41.241802 4946 catalog_manager.cc:4841] TS a5f555d7ab0d43248673885b404448ca (127.18.168.130:39705): tablet 23af3c75848d4077bab974c59df468fb (table test-table [id=8546e39ff25e412188347d5882fe59d8]) successfully deleted
I20260810 18:51:41.242399 5504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:41.242116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:39577 (local address 127.0.0.1:55296)
0810 18:51:41.242242 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:41.242318 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.129:39577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260810 18:51:41.242390 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 5188
W20260810 18:51:41.243919 4945 catalog_manager.cc:4584] TS 7ab2371396004cfc840d46f847dfd2f3 (127.18.168.129:39577): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 23af3c75848d4077bab974c59df468fb: Network error: Client connection negotiation failed: client connection to 127.18.168.129:39577: connect: Connection refused (error 111)
I20260810 18:51:41.247000 5534 ts_tablet_manager.cc:1918] T 23af3c75848d4077bab974c59df468fb P 89fb620a5220480c9887a6a0d6827f19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:51:41.247130 5534 log.cc:1196] T 23af3c75848d4077bab974c59df468fb P 89fb620a5220480c9887a6a0d6827f19: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/wal/wals/23af3c75848d4077bab974c59df468fb
I20260810 18:51:41.247515 5534 ts_tablet_manager.cc:1939] T 23af3c75848d4077bab974c59df468fb P 89fb620a5220480c9887a6a0d6827f19: Deleting consensus metadata
I20260810 18:51:41.248875 4945 catalog_manager.cc:4841] TS 89fb620a5220480c9887a6a0d6827f19 (127.18.168.131:43805): tablet 23af3c75848d4077bab974c59df468fb (table test-table [id=8546e39ff25e412188347d5882fe59d8]) successfully deleted
I20260810 18:51:41.253015 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 5342
I20260810 18:51:41.263688 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 4925
I20260810 18:52:01.273262 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:41883
--webserver_interface=127.18.168.190
--webserver_port=46135
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:33555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:41883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:52:01.386010 5537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:01.386260 5537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:01.386297 5537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:01.390127 5537 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 18:52:01.390177 5537 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 18:52:01.390195 5537 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 18:52:01.390213 5537 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:52:01.390232 5537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:01.390252 5537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:01.390269 5537 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:52:01.390286 5537 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:52:01.395282 5537 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:33555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:41883
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:41883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=46135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:01.396235 5537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:01.397233 5537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:01.405833 5546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:01.405891 5537 server_base.cc:1034] running on GCE node
W20260810 18:52:01.405836 5542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:01.405818 5544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:01.406625 5537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:01.407658 5537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:01.408797 5537 hybrid_clock.cc:648] HybridClock initialized: now 1786387921408775 us; error 31 us; skew 500 ppm
Aug 10 18:52:01 dist-test-slave-cbsf krb5kdc[4287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387921, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:01.411787 5537 init.cc:377] Logged in from keytab as kudu/127.18.168.190@KRBTEST.COM (short username kudu)
I20260810 18:52:01.412683 5537 webserver.cc:460] Webserver started at http://127.18.168.190:46135/ using document root <none> and password file <none>
I20260810 18:52:01.413249 5537 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:01.413339 5537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:01.416963 5537 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.003s
I20260810 18:52:01.419106 5554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:01.420195 5537 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:52:01.420328 5537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "82542fe70878437fb5293cf40bd85162"
format_stamp: "Formatted at 2026-08-10 18:51:18 on dist-test-slave-cbsf"
server_key: "fccb38ac37c35ca792f1674f0133c448"
server_key_iv: "8f470552633b7aaceb5fb7f80da274da"
server_key_version: "encryptionkey@0"
I20260810 18:52:01.420928 5537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:01.434303 5537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:01.437505 5537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:01.437721 5537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:01.445060 5537 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:41883
I20260810 18:52:01.445346 5612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:41883 every 8 connection(s)
I20260810 18:52:01.446213 5537 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:52:01.450628 5613 sys_catalog.cc:263] Verifying existing consensus state
I20260810 18:52:01.450733 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 5537
I20260810 18:52:01.451226 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.129:39577
--local_ip_for_outbound_sockets=127.18.168.129
--tserver_master_addrs=127.18.168.190:41883
--webserver_port=40433
--webserver_interface=127.18.168.129
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:33555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:52:01.453009 5613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162: Bootstrap starting.
I20260810 18:52:01.471314 5613 log.cc:824] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:01.482396 5613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:52:01.482779 5613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162: Bootstrap complete.
I20260810 18:52:01.486933 5613 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82542fe70878437fb5293cf40bd85162" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41883 } }
I20260810 18:52:01.487334 5613 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82542fe70878437fb5293cf40bd85162, State: Initialized, Role: FOLLOWER
I20260810 18:52:01.487790 5613 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82542fe70878437fb5293cf40bd85162" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41883 } }
I20260810 18:52:01.487926 5613 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:52:01.488013 5613 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:52:01.488137 5613 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [term 2 FOLLOWER]: Advancing to term 3
I20260810 18:52:01.490023 5613 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82542fe70878437fb5293cf40bd85162" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41883 } }
I20260810 18:52:01.490350 5613 leader_election.cc:302] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82542fe70878437fb5293cf40bd85162; no voters:
I20260810 18:52:01.490682 5613 leader_election.cc:288] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 18:52:01.490764 5617 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 18:52:01.490942 5617 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [term 3 LEADER]: Becoming Leader. State: Replica: 82542fe70878437fb5293cf40bd85162, State: Running, Role: LEADER
I20260810 18:52:01.491222 5617 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82542fe70878437fb5293cf40bd85162" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41883 } }
I20260810 18:52:01.491891 5613 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:52:01.492115 5619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "82542fe70878437fb5293cf40bd85162" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82542fe70878437fb5293cf40bd85162" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41883 } } }
I20260810 18:52:01.492228 5619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:01.492899 5620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82542fe70878437fb5293cf40bd85162. Latest consensus state: current_term: 3 leader_uuid: "82542fe70878437fb5293cf40bd85162" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82542fe70878437fb5293cf40bd85162" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41883 } } }
I20260810 18:52:01.493007 5620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162 [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:01.493640 5622 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:52:01.498744 5622 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:52:01.499408 5622 catalog_manager.cc:1259] Loaded cluster ID: 00f351aa82bc47d084c4fd8596cfdfe6
I20260810 18:52:01.499563 5622 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:52:01.502262 5622 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:52:01.503090 5622 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162: Loaded TSK: 0
I20260810 18:52:01.503223 5622 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 82542fe70878437fb5293cf40bd85162: Loaded TSK: 1
F20260810 18:52:01.504021 5622 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f169a74d9ad google::LogMessage::Fail() at ??:0
@ 0x7f169a75088a google::LogMessage::SendToLog() at ??:0
@ 0x7f169a74d4de google::LogMessage::Flush() at ??:0
@ 0x7f169a74ea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f169d41118d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f169d409f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f16a50b27bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f16a508c99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f16a508d178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f16a50cedf8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f16a50ed82b std::function<>::operator()() at ??:0
@ 0x7f16a508cb4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f16a508df3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f16a508a579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f16a50ce640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f169c99f3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f169c99fc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f169c9a13f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f169c990512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f169dfd46db start_thread at ??:0
@ 0x7f169924b61f clone at ??:0
W20260810 18:52:01.597584 5615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:01.597854 5615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:01.597975 5615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:01.601938 5615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:01.602025 5615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:01.602139 5615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.129
I20260810 18:52:01.607322 5615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:33555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.129:39577
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.168.129
--webserver_port=40433
--tserver_master_addrs=127.18.168.190:41883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.129
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:01.608371 5615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:01.609442 5615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:01.619375 5665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:01.619381 5667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:01.619740 5669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:01.620087 5615 server_base.cc:1034] running on GCE node
I20260810 18:52:01.620493 5615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:01.621333 5615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:01.622545 5615 hybrid_clock.cc:648] HybridClock initialized: now 1786387921622519 us; error 33 us; skew 500 ppm
Aug 10 18:52:01 dist-test-slave-cbsf krb5kdc[4287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387921, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:01.625790 5615 init.cc:377] Logged in from keytab as kudu/127.18.168.129@KRBTEST.COM (short username kudu)
I20260810 18:52:01.626729 5615 webserver.cc:460] Webserver started at http://127.18.168.129:40433/ using document root <none> and password file <none>
I20260810 18:52:01.627291 5615 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:01.627357 5615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:01.631430 5615 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:52:01.633879 5676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:01.635182 5615 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:52:01.635321 5615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/wal
uuid: "7ab2371396004cfc840d46f847dfd2f3"
format_stamp: "Formatted at 2026-08-10 18:51:19 on dist-test-slave-cbsf"
server_key: "3e3eab977583fb2a067fc976e18dc135"
server_key_iv: "38133292ddfdaec26ef612aa5eaef318"
server_key_version: "encryptionkey@0"
I20260810 18:52:01.635869 5615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:01.651260 5615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:01.654734 5615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:01.654974 5615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:01.655576 5615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:52:01.656730 5685 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 18:52:01.662320 5615 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 18:52:01.662411 5615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260810 18:52:01.662492 5615 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 18:52:01.664248 5615 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 18:52:01.664324 5615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260810 18:52:01.664606 5685 tablet_bootstrap.cc:492] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3: Bootstrap starting.
I20260810 18:52:01.676465 5615 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.129:39577
I20260810 18:52:01.676720 5814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.129:39577 every 8 connection(s)
I20260810 18:52:01.677580 5615 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
I20260810 18:52:01.679466 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 5615
I20260810 18:52:01.679957 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.130:39705
--local_ip_for_outbound_sockets=127.18.168.130
--tserver_master_addrs=127.18.168.190:41883
--webserver_port=44477
--webserver_interface=127.18.168.130
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:33555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:52:01.688769 5685 log.cc:824] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:01.698470 5685 tablet_bootstrap.cc:492] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:52:01.698966 5685 tablet_bootstrap.cc:492] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3: Bootstrap complete.
I20260810 18:52:01.700899 5685 ts_tablet_manager.cc:1397] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3: Time spent bootstrapping tablet: real 0.037s user 0.022s sys 0.014s
I20260810 18:52:01.704403 5685 raft_consensus.cc:348] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fb620a5220480c9887a6a0d6827f19" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43805 } } peers { permanent_uuid: "a5f555d7ab0d43248673885b404448ca" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39705 } } peers { permanent_uuid: "7ab2371396004cfc840d46f847dfd2f3" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39577 } }
I20260810 18:52:01.705039 5685 raft_consensus.cc:729] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ab2371396004cfc840d46f847dfd2f3, State: Initialized, Role: FOLLOWER
I20260810 18:52:01.705658 5685 consensus_queue.cc:260] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fb620a5220480c9887a6a0d6827f19" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43805 } } peers { permanent_uuid: "a5f555d7ab0d43248673885b404448ca" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39705 } } peers { permanent_uuid: "7ab2371396004cfc840d46f847dfd2f3" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39577 } }
I20260810 18:52:01.706908 5685 ts_tablet_manager.cc:1428] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
W20260810 18:52:01.755580 5817 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:52:01.679849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.129:53939)
0810 18:52:01.681256 (+ 1407us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:01.681317 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:01.682492 (+ 1175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:01.754628 (+ 72136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":1164,"thread_start_us":157,"threads_started":1}
W20260810 18:52:01.756213 5815 heartbeater.cc:644] Failed to heartbeat to 127.18.168.190:41883 (0 consecutive failures): Network error: Failed to ping master at 127.18.168.190:41883: Client connection negotiation failed: client connection to 127.18.168.190:41883: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 18:52:01.757067 5817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:01.756740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.129:37415)
0810 18:52:01.756921 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:01.757005 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260810 18:52:01.757814 5817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:01.757610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.129:45637)
0810 18:52:01.757719 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:01.757765 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 18:52:01.758091 5815 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 18:52:01.795660 5820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:01.795923 5820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:01.796000 5820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:01.799988 5820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:01.800079 5820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:01.800184 5820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.130
I20260810 18:52:01.805292 5820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:33555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.130:39705
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.168.130
--webserver_port=44477
--tserver_master_addrs=127.18.168.190:41883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.130
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:01.806322 5820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:01.807302 5820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:01.816912 5831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:01.816947 5830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:01.816947 5833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:01.817137 5820 server_base.cc:1034] running on GCE node
I20260810 18:52:01.817603 5820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:01.818252 5820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:01.819442 5820 hybrid_clock.cc:648] HybridClock initialized: now 1786387921819431 us; error 34 us; skew 500 ppm
Aug 10 18:52:01 dist-test-slave-cbsf krb5kdc[4287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387921, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:01.822240 5820 init.cc:377] Logged in from keytab as kudu/127.18.168.130@KRBTEST.COM (short username kudu)
I20260810 18:52:01.823088 5820 webserver.cc:460] Webserver started at http://127.18.168.130:44477/ using document root <none> and password file <none>
I20260810 18:52:01.823647 5820 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:01.823740 5820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:01.827488 5820 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:52:01.829506 5840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:01.830682 5820 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:52:01.830814 5820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/wal
uuid: "a5f555d7ab0d43248673885b404448ca"
format_stamp: "Formatted at 2026-08-10 18:51:19 on dist-test-slave-cbsf"
server_key: "04b6d50707eb5d13c4bae123288f2742"
server_key_iv: "4d89f223ce06e066b64872d6da3489e2"
server_key_version: "encryptionkey@0"
I20260810 18:52:01.831336 5820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:01.860239 5820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:01.863256 5820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:01.863476 5820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:01.864101 5820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:52:01.865041 5820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:52:01.865127 5820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:01.865199 5820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:52:01.865249 5820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:01.874611 5820 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.130:39705
I20260810 18:52:01.874637 5973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.130:39705 every 8 connection(s)
I20260810 18:52:01.875610 5820 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
I20260810 18:52:01.877990 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 5820
I20260810 18:52:01.878507 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.131:43805
--local_ip_for_outbound_sockets=127.18.168.131
--tserver_master_addrs=127.18.168.190:41883
--webserver_port=42121
--webserver_interface=127.18.168.131
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:33555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:52:01.879739 5976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:01.877805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.130:36745)
0810 18:52:01.878350 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:01.878455 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":154,"threads_started":1}
W20260810 18:52:01.880318 5974 heartbeater.cc:644] Failed to heartbeat to 127.18.168.190:41883 (0 consecutive failures): Network error: Failed to ping master at 127.18.168.190:41883: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
I20260810 18:52:01.881006 5976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:01.880797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.130:59133)
0810 18:52:01.880903 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:01.880952 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260810 18:52:01.881739 5976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:01.881460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.130:56625)
0810 18:52:01.881602 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:01.881663 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 18:52:01.881929 5974 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 18:52:01.990659 5977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:01.990913 5977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:01.990957 5977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:01.995030 5977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:01.995082 5977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:01.995136 5977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.131
I20260810 18:52:02.000177 5977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:33555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.131:43805
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.168.131
--webserver_port=42121
--tserver_master_addrs=127.18.168.190:41883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.131
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:02.001197 5977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:02.002229 5977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:02.011962 5984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:02.011936 5986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:02.011965 5983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:02.012202 5977 server_base.cc:1034] running on GCE node
I20260810 18:52:02.012660 5977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:02.013368 5977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:02.014559 5977 hybrid_clock.cc:648] HybridClock initialized: now 1786387922014544 us; error 27 us; skew 500 ppm
Aug 10 18:52:02 dist-test-slave-cbsf krb5kdc[4287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387922, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:02.017369 5977 init.cc:377] Logged in from keytab as kudu/127.18.168.131@KRBTEST.COM (short username kudu)
I20260810 18:52:02.018162 5977 webserver.cc:460] Webserver started at http://127.18.168.131:42121/ using document root <none> and password file <none>
I20260810 18:52:02.018682 5977 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:02.018740 5977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:02.022396 5977 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:52:02.024328 5994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:02.025575 5977 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:52:02.025691 5977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/wal
uuid: "89fb620a5220480c9887a6a0d6827f19"
format_stamp: "Formatted at 2026-08-10 18:51:19 on dist-test-slave-cbsf"
server_key: "8e2ec631dcc193d64d1d723ed5621860"
server_key_iv: "13c7ef17bdfe814e892f79c8a14fc294"
server_key_version: "encryptionkey@0"
I20260810 18:52:02.026211 5977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:02.042624 5977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:02.045680 5977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:02.045877 5977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:02.046459 5977 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:52:02.047232 5977 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:52:02.047277 5977 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:02.047344 5977 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:52:02.047403 5977 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:02.056188 5977 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.131:43805
I20260810 18:52:02.056260 6133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.131:43805 every 8 connection(s)
I20260810 18:52:02.057227 5977 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
I20260810 18:52:02.061137 6137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:02.059342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.131:57093)
0810 18:52:02.059791 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:02.059895 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":123,"threads_started":1}
W20260810 18:52:02.061601 6134 heartbeater.cc:644] Failed to heartbeat to 127.18.168.190:41883 (0 consecutive failures): Network error: Failed to ping master at 127.18.168.190:41883: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
I20260810 18:52:02.062213 6137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:02.061983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.131:54811)
0810 18:52:02.062101 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:02.062152 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 18:52:02.062786 6137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:02.062584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.131:49193)
0810 18:52:02.062688 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:02.062732 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260810 18:52:02.062956 6134 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 18:52:02.066422 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 5977
I20260810 18:52:02.759150 6139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:02.758669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.129:59431)
0810 18:52:02.758982 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:02.759051 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":85,"threads_started":1}
I20260810 18:52:02.882870 6140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:02.882462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.130:49701)
0810 18:52:02.882705 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:02.882758 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":85,"threads_started":1}
I20260810 18:52:02.996462 6141 raft_consensus.cc:482] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:52:02.996748 6141 raft_consensus.cc:504] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fb620a5220480c9887a6a0d6827f19" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43805 } } peers { permanent_uuid: "a5f555d7ab0d43248673885b404448ca" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39705 } } peers { permanent_uuid: "7ab2371396004cfc840d46f847dfd2f3" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39577 } }
I20260810 18:52:02.998127 6141 leader_election.cc:288] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 89fb620a5220480c9887a6a0d6827f19 (127.18.168.131:43805), a5f555d7ab0d43248673885b404448ca (127.18.168.130:39705)
Aug 10 18:52:03 dist-test-slave-cbsf krb5kdc[4287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387921, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for kudu/127.18.168.130@KRBTEST.COM
Aug 10 18:52:03 dist-test-slave-cbsf krb5kdc[4287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387921, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for kudu/127.18.168.131@KRBTEST.COM
I20260810 18:52:03.012616 6144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:02.998891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:49175 (local address 127.18.168.130:39705)
0810 18:52:02.999199 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:02.999222 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:02.999257 (+ 35us) server_negotiation.cc:407] Connection header received
0810 18:52:02.999926 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:02.999931 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:03.000245 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:03.000566 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:03.001862 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:03.002944 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:03.004275 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:03.004846 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:03.005000 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:03.007600 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:03.007605 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:03.007617 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:03.007643 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:03.009974 (+ 2331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:03.010573 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:03.010577 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:03.010581 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:03.010693 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:03.011080 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:03.011085 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:03.011087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:03.011490 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:03.011633 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:03.012183 (+ 550us) server_negotiation.cc:299] Negotiation successful
0810 18:52:03.012438 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":94,"threads_started":1}
I20260810 18:52:03.012603 6142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:02.998388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:39705 (local address 127.18.168.129:49175)
0810 18:52:02.998761 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:02.998773 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:02.998852 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:03.000604 (+ 1752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:03.000610 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:03.001197 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:52:03.001686 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:03.001701 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:03.003113 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:03.003116 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:03.004147 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:03.004158 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:03.004971 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:03.004974 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:03.005108 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:03.005801 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:52:03.005818 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:52:03.007356 (+ 1538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:52:03.010123 (+ 2767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:03.010128 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:03.010139 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:03.010410 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:03.010811 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:03.010816 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:03.010818 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:03.010961 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:03.011618 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:52:03.011625 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:52:03.011861 (+ 236us) client_negotiation.cc:769] Sending connection context
0810 18:52:03.012051 (+ 190us) client_negotiation.cc:240] Negotiation successful
0810 18:52:03.012411 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"mutex_wait_us":125,"spinlock_wait_cycles":16384,"thread_start_us":76,"threads_started":1}
I20260810 18:52:03.012648 6143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:02.998536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:58577 (local address 127.18.168.131:43805)
0810 18:52:02.999243 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:02.999254 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:02.999288 (+ 34us) server_negotiation.cc:407] Connection header received
0810 18:52:02.999907 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:02.999913 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:03.000245 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:03.000570 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:03.001863 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:03.003023 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:03.004782 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:03.005442 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:03.006031 (+ 589us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:03.007943 (+ 1912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:03.007952 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:03.007970 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:03.008020 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:03.010210 (+ 2190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:03.010696 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:03.010702 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:03.010707 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:03.010842 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:03.011134 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:03.011139 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:03.011140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:03.011497 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:03.011639 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:03.012169 (+ 530us) server_negotiation.cc:299] Negotiation successful
0810 18:52:03.012459 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":79,"threads_started":1}
I20260810 18:52:03.012663 6139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:02.998322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.131:43805 (local address 127.18.168.129:58577)
0810 18:52:02.998519 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:02.998534 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:02.998656 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:03.000604 (+ 1948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:03.000610 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:03.001196 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:52:03.001686 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:03.001700 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:03.003199 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:03.003202 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:03.004149 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:03.004160 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:03.005596 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:03.005601 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:03.005708 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:03.006142 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:52:03.006156 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:52:03.007737 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:52:03.010351 (+ 2614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:03.010355 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:03.010357 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:03.010579 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:03.010945 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:03.010948 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:03.010950 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:03.011038 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:03.011617 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:52:03.011624 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:52:03.011861 (+ 237us) client_negotiation.cc:769] Sending connection context
0810 18:52:03.012050 (+ 189us) client_negotiation.cc:240] Negotiation successful
0810 18:52:03.012499 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":309,"spinlock_wait_cycles":36352}
W20260810 18:52:03.015324 5682 leader_election.cc:341] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a5f555d7ab0d43248673885b404448ca (127.18.168.130:39705): Not found: Tablet not found: 23af3c75848d4077bab974c59df468fb
W20260810 18:52:03.015484 5680 leader_election.cc:341] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 89fb620a5220480c9887a6a0d6827f19 (127.18.168.131:43805): Not found: Tablet not found: 23af3c75848d4077bab974c59df468fb
I20260810 18:52:03.015623 5680 leader_election.cc:302] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ab2371396004cfc840d46f847dfd2f3; no voters: 89fb620a5220480c9887a6a0d6827f19, a5f555d7ab0d43248673885b404448ca
I20260810 18:52:03.015897 6141 raft_consensus.cc:2726] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 18:52:03.063980 6145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:03.063562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.131:58251)
0810 18:52:03.063824 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:03.063882 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":74,"threads_started":1}
I20260810 18:52:03.760339 6146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:03.759904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.129:49675)
0810 18:52:03.760173 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:03.760238 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":92,"threads_started":1}
I20260810 18:52:03.884286 6147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:03.883791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.130:45289)
0810 18:52:03.884112 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:03.884172 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":99,"threads_started":1}
I20260810 18:52:04.065378 6148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:04.064933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.131:59173)
0810 18:52:04.065196 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:04.065255 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":77,"threads_started":1}
I20260810 18:52:04.761559 6151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:04.761127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.129:51387)
0810 18:52:04.761386 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:04.761447 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":74,"threads_started":1}
I20260810 18:52:04.885488 6152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:04.885052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.130:39059)
0810 18:52:04.885331 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:04.885391 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":89,"threads_started":1}
I20260810 18:52:05.066562 6154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:05.066148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.131:52775)
0810 18:52:05.066402 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:05.066458 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":88,"threads_started":1}
I20260810 18:52:05.114777 6155 raft_consensus.cc:482] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:52:05.114914 6155 raft_consensus.cc:504] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fb620a5220480c9887a6a0d6827f19" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43805 } } peers { permanent_uuid: "a5f555d7ab0d43248673885b404448ca" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39705 } } peers { permanent_uuid: "7ab2371396004cfc840d46f847dfd2f3" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39577 } }
I20260810 18:52:05.115386 6155 leader_election.cc:288] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 89fb620a5220480c9887a6a0d6827f19 (127.18.168.131:43805), a5f555d7ab0d43248673885b404448ca (127.18.168.130:39705)
W20260810 18:52:05.116257 5680 leader_election.cc:341] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 89fb620a5220480c9887a6a0d6827f19 (127.18.168.131:43805): Not found: Tablet not found: 23af3c75848d4077bab974c59df468fb
W20260810 18:52:05.116350 5682 leader_election.cc:341] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a5f555d7ab0d43248673885b404448ca (127.18.168.130:39705): Not found: Tablet not found: 23af3c75848d4077bab974c59df468fb
I20260810 18:52:05.116449 5682 leader_election.cc:302] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ab2371396004cfc840d46f847dfd2f3; no voters: 89fb620a5220480c9887a6a0d6827f19, a5f555d7ab0d43248673885b404448ca
I20260810 18:52:05.116629 6155 raft_consensus.cc:2726] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 18:52:05.762692 6156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:05.762264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.129:34011)
0810 18:52:05.762544 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:05.762599 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":91,"threads_started":1}
I20260810 18:52:05.886631 6157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:05.886233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.130:48781)
0810 18:52:05.886477 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:05.886547 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":82,"threads_started":1}
I20260810 18:52:06.067791 6158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:06.067301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.131:52023)
0810 18:52:06.067635 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:06.067697 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":89,"threads_started":1}
I20260810 18:52:06.764039 6159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:06.763613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.129:55073)
0810 18:52:06.763880 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:06.763931 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":73,"threads_started":1}
I20260810 18:52:06.887758 6160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:06.887351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.130:44409)
0810 18:52:06.887620 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:06.887673 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":97,"threads_started":1}
I20260810 18:52:07.068990 6162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:07.068562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.131:48755)
0810 18:52:07.068846 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:07.068902 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":92,"threads_started":1}
I20260810 18:52:07.143864 6163 raft_consensus.cc:482] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:52:07.144019 6163 raft_consensus.cc:504] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fb620a5220480c9887a6a0d6827f19" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43805 } } peers { permanent_uuid: "a5f555d7ab0d43248673885b404448ca" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39705 } } peers { permanent_uuid: "7ab2371396004cfc840d46f847dfd2f3" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39577 } }
I20260810 18:52:07.144553 6163 leader_election.cc:288] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 89fb620a5220480c9887a6a0d6827f19 (127.18.168.131:43805), a5f555d7ab0d43248673885b404448ca (127.18.168.130:39705)
W20260810 18:52:07.145373 5680 leader_election.cc:341] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 89fb620a5220480c9887a6a0d6827f19 (127.18.168.131:43805): Not found: Tablet not found: 23af3c75848d4077bab974c59df468fb
W20260810 18:52:07.145474 5682 leader_election.cc:341] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a5f555d7ab0d43248673885b404448ca (127.18.168.130:39705): Not found: Tablet not found: 23af3c75848d4077bab974c59df468fb
I20260810 18:52:07.145574 5682 leader_election.cc:302] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ab2371396004cfc840d46f847dfd2f3; no voters: 89fb620a5220480c9887a6a0d6827f19, a5f555d7ab0d43248673885b404448ca
I20260810 18:52:07.145730 6163 raft_consensus.cc:2726] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 18:52:07.765219 6164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:07.764791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.129:33939)
0810 18:52:07.765052 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:07.765105 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":79,"threads_started":1}
I20260810 18:52:07.888973 6165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:07.888512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.130:51913)
0810 18:52:07.888832 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:07.888885 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":106,"threads_started":1}
I20260810 18:52:08.070187 6167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:08.069768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.131:40483)
0810 18:52:08.070035 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:08.070094 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":80,"threads_started":1}
I20260810 18:52:08.766407 6168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:08.765979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.129:44461)
0810 18:52:08.766255 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:08.766312 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":93,"threads_started":1}
I20260810 18:52:08.890152 6169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:08.889701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.130:48327)
0810 18:52:08.889989 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:08.890059 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":72,"threads_started":1}
I20260810 18:52:09.071768 6170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:09.071096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.131:43745)
0810 18:52:09.071558 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:09.071633 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":133,"threads_started":1}
I20260810 18:52:09.767644 6171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:09.767188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.129:37347)
0810 18:52:09.767454 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:09.767511 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":95,"threads_started":1}
I20260810 18:52:09.891229 6172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:09.890831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.130:36609)
0810 18:52:09.891085 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:09.891139 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":90,"threads_started":1}
I20260810 18:52:10.073053 6173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:10.072608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.131:37045)
0810 18:52:10.072888 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:10.072940 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":126,"threads_started":1}
I20260810 18:52:10.768913 6174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:10.768449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.129:41121)
0810 18:52:10.768758 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:10.768816 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":80,"threads_started":1}
I20260810 18:52:10.892306 6175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:10.891916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.130:56077)
0810 18:52:10.892167 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:10.892217 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":86,"threads_started":1}
I20260810 18:52:11.074257 6176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:11.073793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.131:32891)
0810 18:52:11.074056 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:11.074143 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":94,"threads_started":1}
I20260810 18:52:11.770216 6177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:11.769815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.129:51881)
0810 18:52:11.770064 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:11.770121 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":74,"threads_started":1}
I20260810 18:52:11.893388 6178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:11.893010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.130:44537)
0810 18:52:11.893249 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:11.893299 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":85,"threads_started":1}
I20260810 18:52:12.075542 6179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:12.075112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.131:47071)
0810 18:52:12.075389 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:12.075444 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":101,"threads_started":1}
I20260810 18:52:12.200129 6180 raft_consensus.cc:482] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:52:12.200289 6180 raft_consensus.cc:504] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fb620a5220480c9887a6a0d6827f19" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43805 } } peers { permanent_uuid: "a5f555d7ab0d43248673885b404448ca" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39705 } } peers { permanent_uuid: "7ab2371396004cfc840d46f847dfd2f3" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39577 } }
I20260810 18:52:12.200775 6180 leader_election.cc:288] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 89fb620a5220480c9887a6a0d6827f19 (127.18.168.131:43805), a5f555d7ab0d43248673885b404448ca (127.18.168.130:39705)
W20260810 18:52:12.201581 5680 leader_election.cc:341] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 89fb620a5220480c9887a6a0d6827f19 (127.18.168.131:43805): Not found: Tablet not found: 23af3c75848d4077bab974c59df468fb
W20260810 18:52:12.201699 5682 leader_election.cc:341] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a5f555d7ab0d43248673885b404448ca (127.18.168.130:39705): Not found: Tablet not found: 23af3c75848d4077bab974c59df468fb
I20260810 18:52:12.201838 5682 leader_election.cc:302] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ab2371396004cfc840d46f847dfd2f3; no voters: 89fb620a5220480c9887a6a0d6827f19, a5f555d7ab0d43248673885b404448ca
I20260810 18:52:12.201998 6180 raft_consensus.cc:2726] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 18:52:12.771384 6181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:12.770967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.129:49857)
0810 18:52:12.771231 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:12.771285 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":96,"threads_started":1}
I20260810 18:52:12.894496 6182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:12.894092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.130:40679)
0810 18:52:12.894324 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:12.894375 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":69,"threads_started":1}
I20260810 18:52:13.076733 6183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:13.076267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.131:36341)
0810 18:52:13.076549 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:13.076620 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":90,"threads_started":1}
I20260810 18:52:13.772553 6184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:13.772159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.129:38797)
0810 18:52:13.772405 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:13.772456 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":83,"threads_started":1}
I20260810 18:52:13.895601 6185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:13.895193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.130:48133)
0810 18:52:13.895446 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:13.895505 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":73,"threads_started":1}
I20260810 18:52:14.077839 6186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:14.077447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.131:34009)
0810 18:52:14.077690 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:14.077744 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":77,"threads_started":1}
I20260810 18:52:14.773833 6187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:14.773394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.129:51047)
0810 18:52:14.773681 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:14.773734 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":79,"threads_started":1}
I20260810 18:52:14.896926 6188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:14.896364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.130:34083)
0810 18:52:14.896737 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:14.896804 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":112,"threads_started":1}
I20260810 18:52:15.079130 6189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:15.078681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.131:43049)
0810 18:52:15.078964 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:15.079026 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":99,"threads_started":1}
I20260810 18:52:15.775203 6191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:15.774653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.129:54015)
0810 18:52:15.775024 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:15.775101 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":97,"threads_started":1}
I20260810 18:52:15.898571 6192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:15.897916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.130:33659)
0810 18:52:15.898319 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:15.898422 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":103,"threads_started":1}
I20260810 18:52:16.080399 6193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:16.079992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.131:44127)
0810 18:52:16.080244 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:16.080304 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":79,"threads_started":1}
I20260810 18:52:16.776772 6194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:16.776274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.129:60323)
0810 18:52:16.776526 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:16.776614 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":76,"threads_started":1}
I20260810 18:52:16.827474 6197 raft_consensus.cc:482] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:52:16.827617 6197 raft_consensus.cc:504] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fb620a5220480c9887a6a0d6827f19" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 43805 } } peers { permanent_uuid: "a5f555d7ab0d43248673885b404448ca" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 39705 } } peers { permanent_uuid: "7ab2371396004cfc840d46f847dfd2f3" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39577 } }
I20260810 18:52:16.828125 6197 leader_election.cc:288] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 89fb620a5220480c9887a6a0d6827f19 (127.18.168.131:43805), a5f555d7ab0d43248673885b404448ca (127.18.168.130:39705)
W20260810 18:52:16.828970 5682 leader_election.cc:341] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a5f555d7ab0d43248673885b404448ca (127.18.168.130:39705): Not found: Tablet not found: 23af3c75848d4077bab974c59df468fb
W20260810 18:52:16.829099 5680 leader_election.cc:341] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 89fb620a5220480c9887a6a0d6827f19 (127.18.168.131:43805): Not found: Tablet not found: 23af3c75848d4077bab974c59df468fb
I20260810 18:52:16.829221 5680 leader_election.cc:302] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ab2371396004cfc840d46f847dfd2f3; no voters: 89fb620a5220480c9887a6a0d6827f19, a5f555d7ab0d43248673885b404448ca
I20260810 18:52:16.829391 6197 raft_consensus.cc:2726] T 23af3c75848d4077bab974c59df468fb P 7ab2371396004cfc840d46f847dfd2f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 18:52:16.899976 6198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:16.899543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.130:59835)
0810 18:52:16.899820 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:16.899879 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":91,"threads_started":1}
I20260810 18:52:17.074375 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 5615
I20260810 18:52:17.081846 6199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:17.081403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:41883 (local address 127.18.168.131:41093)
0810 18:52:17.081680 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:17.081747 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.190:41883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":106,"threads_started":1}
I20260810 18:52:17.084293 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 5820
I20260810 18:52:17.091334 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 5977
2026-08-10T18:52:17Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59764 ms)
[----------] 30 tests from SecurityITest (249230 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:52:17 dist-test-slave-cbsf krb5kdc[6202](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:52:17 dist-test-slave-cbsf krb5kdc[6202](info): set up 2 sockets
Aug 10 18:52:17 dist-test-slave-cbsf krb5kdc[6202](info): commencing operation
krb5kdc: starting...
W20260810 18:52:18.200960 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:52:18 dist-test-slave-cbsf krb5kdc[6202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:52:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:52:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.168.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
I20260810 18:52:18.328984 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:39667
--webserver_interface=127.18.168.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:43183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:39667
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:52:18.438695 6217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:18.438922 6217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:18.438966 6217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:18.442804 6217 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:52:18.442850 6217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:18.442869 6217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:18.442885 6217 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:52:18.442899 6217 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:52:18.447860 6217 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:43183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:39667
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:39667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:18.448796 6217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:18.449723 6217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:18.458048 6224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:18.458081 6222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:18.458117 6229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:18.458196 6217 server_base.cc:1034] running on GCE node
I20260810 18:52:18.458920 6217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:18.460002 6217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:18.461184 6217 hybrid_clock.cc:648] HybridClock initialized: now 1786387938461172 us; error 24 us; skew 500 ppm
Aug 10 18:52:18 dist-test-slave-cbsf krb5kdc[6202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387938, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:18.464048 6217 init.cc:377] Logged in from keytab as kudu/127.18.168.190@KRBTEST.COM (short username kudu)
I20260810 18:52:18.465056 6217 webserver.cc:460] Webserver started at http://127.18.168.190:33727/ using document root <none> and password file <none>
I20260810 18:52:18.465656 6217 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:18.465745 6217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:18.465986 6217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:18.468007 6217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "c77c3890120546419e596619784ad997"
format_stamp: "Formatted at 2026-08-10 18:52:18 on dist-test-slave-cbsf"
server_key: "d08d1f300dbc167c532f86bf6e2d4f54"
server_key_iv: "3042be9fc416d14efa3c303617f4935b"
server_key_version: "encryptionkey@0"
I20260810 18:52:18.468575 6217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "c77c3890120546419e596619784ad997"
format_stamp: "Formatted at 2026-08-10 18:52:18 on dist-test-slave-cbsf"
server_key: "d08d1f300dbc167c532f86bf6e2d4f54"
server_key_iv: "3042be9fc416d14efa3c303617f4935b"
server_key_version: "encryptionkey@0"
I20260810 18:52:18.472123 6217 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 18:52:18.474589 6237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:18.475795 6217 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 18:52:18.475943 6217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "c77c3890120546419e596619784ad997"
format_stamp: "Formatted at 2026-08-10 18:52:18 on dist-test-slave-cbsf"
server_key: "d08d1f300dbc167c532f86bf6e2d4f54"
server_key_iv: "3042be9fc416d14efa3c303617f4935b"
server_key_version: "encryptionkey@0"
I20260810 18:52:18.476050 6217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:18.494117 6217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:18.497169 6217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:18.497382 6217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:18.504822 6217 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:39667
I20260810 18:52:18.504861 6296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:39667 every 8 connection(s)
I20260810 18:52:18.505802 6217 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:52:18.508648 6297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:18.514295 6297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c77c3890120546419e596619784ad997: Bootstrap starting.
I20260810 18:52:18.515353 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 6217
I20260810 18:52:18.515499 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:52:18.515857 19106 external_mini_cluster.cc:1428] Setting key faa7351a27963c567905ac954407657e
I20260810 18:52:18.517105 6297 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c77c3890120546419e596619784ad997: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:18.517943 6297 log.cc:824] T 00000000000000000000000000000000 P c77c3890120546419e596619784ad997: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:18.519874 6297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c77c3890120546419e596619784ad997: No bootstrap required, opened a new log
I20260810 18:52:18.523552 6297 raft_consensus.cc:348] T 00000000000000000000000000000000 P c77c3890120546419e596619784ad997 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c77c3890120546419e596619784ad997" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 39667 } }
I20260810 18:52:18.523854 6297 raft_consensus.cc:374] T 00000000000000000000000000000000 P c77c3890120546419e596619784ad997 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:18.523923 6297 raft_consensus.cc:729] T 00000000000000000000000000000000 P c77c3890120546419e596619784ad997 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c77c3890120546419e596619784ad997, State: Initialized, Role: FOLLOWER
I20260810 18:52:18.524468 6297 consensus_queue.cc:260] T 00000000000000000000000000000000 P c77c3890120546419e596619784ad997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c77c3890120546419e596619784ad997" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 39667 } }
I20260810 18:52:18.524645 6297 raft_consensus.cc:388] T 00000000000000000000000000000000 P c77c3890120546419e596619784ad997 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:52:18.524731 6297 raft_consensus.cc:482] T 00000000000000000000000000000000 P c77c3890120546419e596619784ad997 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:52:18.524832 6297 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c77c3890120546419e596619784ad997 [term 0 FOLLOWER]: Advancing to term 1
Aug 10 18:52:18 dist-test-slave-cbsf krb5kdc[6202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:52:18.525700 6297 raft_consensus.cc:504] T 00000000000000000000000000000000 P c77c3890120546419e596619784ad997 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c77c3890120546419e596619784ad997" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 39667 } }
I20260810 18:52:18.526078 6297 leader_election.cc:302] T 00000000000000000000000000000000 P c77c3890120546419e596619784ad997 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c77c3890120546419e596619784ad997; no voters:
I20260810 18:52:18.526433 6297 leader_election.cc:288] T 00000000000000000000000000000000 P c77c3890120546419e596619784ad997 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:52:18.526507 6302 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c77c3890120546419e596619784ad997 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:52:18.526729 6302 raft_consensus.cc:686] T 00000000000000000000000000000000 P c77c3890120546419e596619784ad997 [term 1 LEADER]: Becoming Leader. State: Replica: c77c3890120546419e596619784ad997, State: Running, Role: LEADER
I20260810 18:52:18.527038 6302 consensus_queue.cc:237] T 00000000000000000000000000000000 P c77c3890120546419e596619784ad997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c77c3890120546419e596619784ad997" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 39667 } }
I20260810 18:52:18.527693 6297 sys_catalog.cc:564] T 00000000000000000000000000000000 P c77c3890120546419e596619784ad997 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:52:18.528251 6302 sys_catalog.cc:455] T 00000000000000000000000000000000 P c77c3890120546419e596619784ad997 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c77c3890120546419e596619784ad997. Latest consensus state: current_term: 1 leader_uuid: "c77c3890120546419e596619784ad997" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c77c3890120546419e596619784ad997" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 39667 } } }
I20260810 18:52:18.528391 6302 sys_catalog.cc:458] T 00000000000000000000000000000000 P c77c3890120546419e596619784ad997 [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:18.528734 6303 sys_catalog.cc:455] T 00000000000000000000000000000000 P c77c3890120546419e596619784ad997 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c77c3890120546419e596619784ad997" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c77c3890120546419e596619784ad997" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 39667 } } }
I20260810 18:52:18.528826 6303 sys_catalog.cc:458] T 00000000000000000000000000000000 P c77c3890120546419e596619784ad997 [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:18.528851 6309 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:52:18.531708 6309 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:52:18.532202 6300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:18.517046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48494 (local address 127.18.168.190:39667)
0810 18:52:18.517532 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:18.517547 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:18.517582 (+ 35us) server_negotiation.cc:407] Connection header received
0810 18:52:18.518370 (+ 788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:18.518395 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:18.518819 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:18.519147 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:18.519948 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:18.521089 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:18.521945 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:18.522428 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:18.522603 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:18.525456 (+ 2853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:18.525468 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:18.525485 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:18.525514 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:18.527566 (+ 2052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:18.528191 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:18.528197 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:18.528203 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:18.528351 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:18.528958 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:18.528962 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:18.528964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:18.529498 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:18.530488 (+ 990us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:18.530689 (+ 201us) server_negotiation.cc:299] Negotiation successful
0810 18:52:18.531000 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":148,"threads_started":1}
I20260810 18:52:18.537596 6309 catalog_manager.cc:1347] Generated new cluster ID: 84b8c8d96f8d49cf8e118a1a557e651a
I20260810 18:52:18.537663 6309 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:52:18.553511 6309 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:52:18.554695 6309 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:52:18.561393 6309 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c77c3890120546419e596619784ad997: Generated new TSK 0
I20260810 18:52:18.562057 6309 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.168.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab.
Entry for principal kudu/127.18.168.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab.
I20260810 18:52:18.595682 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.129:0
--local_ip_for_outbound_sockets=127.18.168.129
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:39667
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:43183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:52:18.707270 6327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:18.707541 6327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:18.707616 6327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:18.711563 6327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:18.711640 6327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:18.711728 6327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.129
I20260810 18:52:18.716841 6327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:43183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:39667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.129
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:18.717849 6327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:18.718840 6327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:18.728379 6335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:18.728483 6332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:18.728379 6333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:18.728662 6327 server_base.cc:1034] running on GCE node
I20260810 18:52:18.729171 6327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:18.729929 6327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:18.731123 6327 hybrid_clock.cc:648] HybridClock initialized: now 1786387938731108 us; error 28 us; skew 500 ppm
Aug 10 18:52:18 dist-test-slave-cbsf krb5kdc[6202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387938, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:18.733986 6327 init.cc:377] Logged in from keytab as kudu/127.18.168.129@KRBTEST.COM (short username kudu)
I20260810 18:52:18.734866 6327 webserver.cc:460] Webserver started at http://127.18.168.129:45225/ using document root <none> and password file <none>
I20260810 18:52:18.735422 6327 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:18.735510 6327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:18.735733 6327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:18.737802 6327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-0/data/instance:
uuid: "6d200bb633c84796aef74f47dc576c69"
format_stamp: "Formatted at 2026-08-10 18:52:18 on dist-test-slave-cbsf"
server_key: "8102efa1a30827ef52051b866b9209c3"
server_key_iv: "2f28fe0464e0885838ae207a95a4c4db"
server_key_version: "encryptionkey@0"
I20260810 18:52:18.738384 6327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance:
uuid: "6d200bb633c84796aef74f47dc576c69"
format_stamp: "Formatted at 2026-08-10 18:52:18 on dist-test-slave-cbsf"
server_key: "8102efa1a30827ef52051b866b9209c3"
server_key_iv: "2f28fe0464e0885838ae207a95a4c4db"
server_key_version: "encryptionkey@0"
I20260810 18:52:18.742144 6327 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:52:18.744522 6345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:18.745815 6327 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:52:18.745955 6327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-0/wal
uuid: "6d200bb633c84796aef74f47dc576c69"
format_stamp: "Formatted at 2026-08-10 18:52:18 on dist-test-slave-cbsf"
server_key: "8102efa1a30827ef52051b866b9209c3"
server_key_iv: "2f28fe0464e0885838ae207a95a4c4db"
server_key_version: "encryptionkey@0"
I20260810 18:52:18.746061 6327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:18.765790 6327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:18.768607 6327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:18.768837 6327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:18.769425 6327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:52:18.770267 6327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:52:18.770339 6327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:18.770409 6327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:52:18.770462 6327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:18.779289 6327 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.129:44587
I20260810 18:52:18.779347 6478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.129:44587 every 8 connection(s)
I20260810 18:52:18.780193 6327 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
I20260810 18:52:18.782193 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 6327
I20260810 18:52:18.782347 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance
I20260810 18:52:18.782666 19106 external_mini_cluster.cc:1428] Setting key ab28c58b89220dc5782f31ac41b823e9
Aug 10 18:52:18 dist-test-slave-cbsf krb5kdc[6202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387938, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.168.130@KRBTEST.COM; defaulting to no policy
I20260810 18:52:18.794710 6300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:18.782108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:51197 (local address 127.18.168.190:39667)
0810 18:52:18.782276 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:18.782280 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:18.783122 (+ 842us) server_negotiation.cc:407] Connection header received
0810 18:52:18.784648 (+ 1526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:18.784651 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:18.784732 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:18.784830 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:18.785967 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:18.786516 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:18.787691 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:18.788080 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:18.788203 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:18.790800 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:18.790805 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:18.790810 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:18.790843 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:18.792527 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:18.793109 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:18.793113 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:18.793115 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:18.793212 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:18.793594 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:18.793600 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:18.793605 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:18.793848 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:18.793966 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:18.794466 (+ 500us) server_negotiation.cc:299] Negotiation successful
0810 18:52:18.794590 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 18:52:18.795518 6481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:18.782493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:39667 (local address 127.18.168.129:51197)
0810 18:52:18.782954 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:18.783006 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:18.784440 (+ 1434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:18.785006 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:18.785013 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:18.785353 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:52:18.785805 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:18.785818 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:18.786650 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:18.786657 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:18.787562 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:18.787572 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:18.788250 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:18.788253 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:18.788404 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:18.789052 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:52:18.789073 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:52:18.790618 (+ 1545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:52:18.792656 (+ 2038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:18.792661 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:18.792673 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:18.793004 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:18.793315 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:18.793317 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:18.793318 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:18.793481 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:18.793966 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:52:18.793970 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:52:18.794188 (+ 218us) client_negotiation.cc:769] Sending connection context
0810 18:52:18.794379 (+ 191us) client_negotiation.cc:240] Negotiation successful
0810 18:52:18.794658 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":140,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.130@KRBTEST.COM" created.
I20260810 18:52:18.796844 6479 heartbeater.cc:347] Connected to a master server at 127.18.168.190:39667
I20260810 18:52:18.798424 6479 heartbeater.cc:464] Registering TS with master...
I20260810 18:52:18.799077 6479 heartbeater.cc:511] Master 127.18.168.190:39667 requested a full tablet report, sending...
I20260810 18:52:18.800837 6256 ts_manager.cc:194] Registered new tserver with Master: 6d200bb633c84796aef74f47dc576c69 (127.18.168.129:44587)
I20260810 18:52:18.802192 6256 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.129@KRBTEST.COM'} at 127.18.168.129:51197
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab.
Entry for principal kudu/127.18.168.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab.
I20260810 18:52:18.816257 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.130:0
--local_ip_for_outbound_sockets=127.18.168.130
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:39667
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:43183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:52:18.931329 6484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:18.931577 6484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:18.931645 6484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:18.936312 6484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:18.936371 6484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:18.936460 6484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.130
I20260810 18:52:18.941864 6484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:43183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:39667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.130
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:18.942976 6484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:18.944054 6484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:18.953883 6492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:18.953872 6490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:18.953872 6489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:18.954170 6484 server_base.cc:1034] running on GCE node
I20260810 18:52:18.954572 6484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:18.955201 6484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:18.956377 6484 hybrid_clock.cc:648] HybridClock initialized: now 1786387938956338 us; error 50 us; skew 500 ppm
Aug 10 18:52:18 dist-test-slave-cbsf krb5kdc[6202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387938, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:18.959167 6484 init.cc:377] Logged in from keytab as kudu/127.18.168.130@KRBTEST.COM (short username kudu)
I20260810 18:52:18.960062 6484 webserver.cc:460] Webserver started at http://127.18.168.130:43401/ using document root <none> and password file <none>
I20260810 18:52:18.960644 6484 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:18.960762 6484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:18.960997 6484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:18.963016 6484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-1/data/instance:
uuid: "1c27ca78b05045cbbc88e4b17860654f"
format_stamp: "Formatted at 2026-08-10 18:52:18 on dist-test-slave-cbsf"
server_key: "0013dba9be4f16fb415a4053bf70d1df"
server_key_iv: "f32737c03ec58f55eea5644a1cf255a3"
server_key_version: "encryptionkey@0"
I20260810 18:52:18.963660 6484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance:
uuid: "1c27ca78b05045cbbc88e4b17860654f"
format_stamp: "Formatted at 2026-08-10 18:52:18 on dist-test-slave-cbsf"
server_key: "0013dba9be4f16fb415a4053bf70d1df"
server_key_iv: "f32737c03ec58f55eea5644a1cf255a3"
server_key_version: "encryptionkey@0"
I20260810 18:52:18.967393 6484 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 18:52:18.969830 6500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:18.971020 6484 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:52:18.971167 6484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-1/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-1/wal
uuid: "1c27ca78b05045cbbc88e4b17860654f"
format_stamp: "Formatted at 2026-08-10 18:52:18 on dist-test-slave-cbsf"
server_key: "0013dba9be4f16fb415a4053bf70d1df"
server_key_iv: "f32737c03ec58f55eea5644a1cf255a3"
server_key_version: "encryptionkey@0"
I20260810 18:52:18.971273 6484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:19.017532 6484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:19.020537 6484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:19.020767 6484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:19.021406 6484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:52:19.022289 6484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:52:19.022359 6484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:19.022430 6484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:52:19.022477 6484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:19.032599 6484 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.130:41551
I20260810 18:52:19.032626 6633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.130:41551 every 8 connection(s)
I20260810 18:52:19.033600 6484 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
I20260810 18:52:19.034380 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 6484
I20260810 18:52:19.034529 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance
I20260810 18:52:19.034821 19106 external_mini_cluster.cc:1428] Setting key 2a39f18394653cd16b706a79955afbf5
Aug 10 18:52:19 dist-test-slave-cbsf krb5kdc[6202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387938, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.168.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.131@KRBTEST.COM" created.
I20260810 18:52:19.049005 6300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:19.036876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:53741 (local address 127.18.168.190:39667)
0810 18:52:19.037037 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:19.037045 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:19.037203 (+ 158us) server_negotiation.cc:407] Connection header received
0810 18:52:19.038538 (+ 1335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:19.038543 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:19.038606 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:19.038719 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:19.039888 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:19.040778 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:19.042022 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:19.042456 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:19.042605 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:19.045102 (+ 2497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:19.045108 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:19.045114 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:19.045147 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:19.046698 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:19.047275 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:19.047278 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:19.047279 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:19.047358 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:19.047686 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:19.047688 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:19.047689 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:19.047855 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:19.047930 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:19.048563 (+ 633us) server_negotiation.cc:299] Negotiation successful
0810 18:52:19.048763 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 18:52:19.050072 6636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:19.035948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:39667 (local address 127.18.168.130:53741)
0810 18:52:19.037065 (+ 1117us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:19.037106 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:19.038322 (+ 1216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:19.038872 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:19.038878 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:19.039223 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:52:19.039741 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:19.039755 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:19.040902 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:19.040905 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:19.041893 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:19.041903 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:19.042623 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:19.042626 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:19.042763 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:19.043390 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:52:19.043406 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:52:19.044933 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:52:19.046846 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:19.046855 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:19.046871 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:19.047180 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:19.047441 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:19.047443 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:19.047445 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:19.047604 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:19.047984 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:52:19.047990 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:52:19.048294 (+ 304us) client_negotiation.cc:769] Sending connection context
0810 18:52:19.048453 (+ 159us) client_negotiation.cc:240] Negotiation successful
0810 18:52:19.048872 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":910,"thread_start_us":141,"threads_started":1}
I20260810 18:52:19.051857 6634 heartbeater.cc:347] Connected to a master server at 127.18.168.190:39667
I20260810 18:52:19.052132 6634 heartbeater.cc:464] Registering TS with master...
I20260810 18:52:19.052661 6634 heartbeater.cc:511] Master 127.18.168.190:39667 requested a full tablet report, sending...
I20260810 18:52:19.053927 6256 ts_manager.cc:194] Registered new tserver with Master: 1c27ca78b05045cbbc88e4b17860654f (127.18.168.130:41551)
I20260810 18:52:19.054836 6256 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.130@KRBTEST.COM'} at 127.18.168.130:53741
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab.
Entry for principal kudu/127.18.168.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab.
I20260810 18:52:19.067416 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.131:0
--local_ip_for_outbound_sockets=127.18.168.131
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:39667
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:43183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:52:19.182726 6639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:19.182960 6639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:19.183001 6639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:19.186913 6639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:19.186967 6639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:19.187021 6639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.131
I20260810 18:52:19.192066 6639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:43183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:39667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.131
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:19.193167 6639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:19.194188 6639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:19.204789 6644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:19.204789 6648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:19.204829 6645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:19.205225 6639 server_base.cc:1034] running on GCE node
I20260810 18:52:19.205719 6639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:19.206385 6639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:19.207602 6639 hybrid_clock.cc:648] HybridClock initialized: now 1786387939207590 us; error 32 us; skew 500 ppm
Aug 10 18:52:19 dist-test-slave-cbsf krb5kdc[6202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387939, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:19.210713 6639 init.cc:377] Logged in from keytab as kudu/127.18.168.131@KRBTEST.COM (short username kudu)
I20260810 18:52:19.211623 6639 webserver.cc:460] Webserver started at http://127.18.168.131:36709/ using document root <none> and password file <none>
I20260810 18:52:19.212229 6639 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:19.212327 6639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:19.212579 6639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:19.214802 6639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-2/data/instance:
uuid: "f98f188e8b5c4abfb39296e38d9dc1cb"
format_stamp: "Formatted at 2026-08-10 18:52:19 on dist-test-slave-cbsf"
server_key: "74793410c6af384792d5f5cbe423fe8a"
server_key_iv: "651a6f5c7f595408d142c3575ff6f5d5"
server_key_version: "encryptionkey@0"
I20260810 18:52:19.215418 6639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance:
uuid: "f98f188e8b5c4abfb39296e38d9dc1cb"
format_stamp: "Formatted at 2026-08-10 18:52:19 on dist-test-slave-cbsf"
server_key: "74793410c6af384792d5f5cbe423fe8a"
server_key_iv: "651a6f5c7f595408d142c3575ff6f5d5"
server_key_version: "encryptionkey@0"
I20260810 18:52:19.219446 6639 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260810 18:52:19.222075 6656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:19.223209 6639 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:52:19.223351 6639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-2/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-2/wal
uuid: "f98f188e8b5c4abfb39296e38d9dc1cb"
format_stamp: "Formatted at 2026-08-10 18:52:19 on dist-test-slave-cbsf"
server_key: "74793410c6af384792d5f5cbe423fe8a"
server_key_iv: "651a6f5c7f595408d142c3575ff6f5d5"
server_key_version: "encryptionkey@0"
I20260810 18:52:19.223464 6639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:19.242602 6639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:19.245697 6639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:19.245893 6639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:19.246462 6639 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:52:19.247319 6639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:52:19.247388 6639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:19.247454 6639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:52:19.247510 6639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:19.257102 6639 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.131:41085
I20260810 18:52:19.257133 6786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.131:41085 every 8 connection(s)
I20260810 18:52:19.257997 6639 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
I20260810 18:52:19.264532 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 6639
I20260810 18:52:19.264686 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance
I20260810 18:52:19.265005 19106 external_mini_cluster.cc:1428] Setting key 5e531e3aec85126db8ffdfe1ce09d4a0
Aug 10 18:52:19 dist-test-slave-cbsf krb5kdc[6202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387939, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:52:19.271811 6300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:19.259753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:56233 (local address 127.18.168.190:39667)
0810 18:52:19.259895 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:19.259900 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:19.260756 (+ 856us) server_negotiation.cc:407] Connection header received
0810 18:52:19.262157 (+ 1401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:19.262160 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:19.262204 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:19.262273 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:19.263355 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:19.263885 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:19.265225 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:19.265615 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:19.265749 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:19.268270 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:19.268274 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:19.268277 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:19.268303 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:19.269791 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:19.270308 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:19.270312 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:19.270314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:19.270398 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:19.270744 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:19.270748 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:19.270749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:19.270928 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:19.271034 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:19.271513 (+ 479us) server_negotiation.cc:299] Negotiation successful
0810 18:52:19.271647 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 18:52:19.272536 6789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:19.260155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:39667 (local address 127.18.168.131:56233)
0810 18:52:19.260581 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:19.260621 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:19.261958 (+ 1337us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:19.262432 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:19.262439 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:19.262758 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:52:19.263212 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:19.263243 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:19.264004 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:19.264006 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:19.265100 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:19.265111 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:19.265788 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:19.265791 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:19.266017 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:19.266604 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:52:19.266620 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:52:19.268116 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:52:19.269918 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:19.269924 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:19.269937 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:19.270199 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:19.270492 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:19.270496 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:19.270498 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:19.270645 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:19.271044 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:52:19.271053 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:52:19.271278 (+ 225us) client_negotiation.cc:769] Sending connection context
0810 18:52:19.271471 (+ 193us) client_negotiation.cc:240] Negotiation successful
0810 18:52:19.271774 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":117,"threads_started":1}
I20260810 18:52:19.273548 6787 heartbeater.cc:347] Connected to a master server at 127.18.168.190:39667
I20260810 18:52:19.273752 6787 heartbeater.cc:464] Registering TS with master...
I20260810 18:52:19.274161 6787 heartbeater.cc:511] Master 127.18.168.190:39667 requested a full tablet report, sending...
I20260810 18:52:19.275051 6256 ts_manager.cc:194] Registered new tserver with Master: f98f188e8b5c4abfb39296e38d9dc1cb (127.18.168.131:41085)
I20260810 18:52:19.275599 6256 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.131@KRBTEST.COM'} at 127.18.168.131:56233
I20260810 18:52:19.278620 19106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:52:19.287937 6300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:19.280768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48504 (local address 127.18.168.190:39667)
0810 18:52:19.280873 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:19.280877 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:19.281134 (+ 257us) server_negotiation.cc:407] Connection header received
0810 18:52:19.281286 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:19.281289 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:19.281328 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:19.281412 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:19.282098 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:19.282593 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:19.283484 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:19.283707 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:19.283811 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:19.285109 (+ 1298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:19.285113 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:19.285115 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:19.285138 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:19.286301 (+ 1163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:19.286727 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:19.286731 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:19.286733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:19.286815 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:19.287091 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:19.287095 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:19.287096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:19.287313 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:19.287446 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:19.287673 (+ 227us) server_negotiation.cc:299] Negotiation successful
0810 18:52:19.287790 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 18:52:19.290488 6256 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48504:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:52:19.293460 6256 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:52:19.307696 6801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:19.301496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.131:41085 (local address 127.0.0.1:46926)
0810 18:52:19.301958 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:19.302011 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:19.302136 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:19.302808 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:19.302817 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:19.302882 (+ 65us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:19.303026 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:19.303034 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:19.304440 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:19.304443 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:19.305787 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:19.305798 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:19.307102 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:19.307107 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:19.307216 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:19.307261 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 18:52:19.307362 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 18:52:19.307542 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":23,"thread_start_us":78,"threads_started":1}
I20260810 18:52:19.308024 6802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:19.301636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46926 (local address 127.18.168.131:41085)
0810 18:52:19.302124 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:19.302136 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:19.302160 (+ 24us) server_negotiation.cc:407] Connection header received
0810 18:52:19.302325 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:19.302331 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:19.302542 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:19.302694 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:19.303175 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:19.304295 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:19.306048 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:19.306969 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:19.307106 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:19.307256 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:19.307779 (+ 523us) server_negotiation.cc:299] Negotiation successful
0810 18:52:19.307875 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":164,"threads_started":1}
I20260810 18:52:19.308102 6806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:19.302811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43514 (local address 127.18.168.129:44587)
0810 18:52:19.303152 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:19.303159 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:19.303181 (+ 22us) server_negotiation.cc:407] Connection header received
0810 18:52:19.303257 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:19.303265 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:19.303443 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:19.303578 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:19.304051 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:19.304993 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:19.306328 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:19.307122 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:19.307230 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:19.307345 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:19.307619 (+ 274us) server_negotiation.cc:299] Negotiation successful
0810 18:52:19.307731 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":115,"threads_started":1}
I20260810 18:52:19.308195 6798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:19.301603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:41551 (local address 127.0.0.1:51680)
0810 18:52:19.302174 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:19.302187 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:19.302268 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:19.303924 (+ 1656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:19.303927 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:19.303944 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:19.304031 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:19.304036 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:19.305350 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:19.305355 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:19.306470 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:19.306482 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:19.307576 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:19.307580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:19.307663 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:19.307683 (+ 20us) client_negotiation.cc:769] Sending connection context
0810 18:52:19.307755 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 18:52:19.308038 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"mutex_wait_us":3,"thread_start_us":85,"threads_started":1}
I20260810 18:52:19.308665 6805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:19.302635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51680 (local address 127.18.168.130:41551)
0810 18:52:19.303370 (+ 735us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:19.303385 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:19.303408 (+ 23us) server_negotiation.cc:407] Connection header received
0810 18:52:19.303481 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:19.303486 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:19.303661 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:19.303796 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:19.304289 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:19.305206 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:19.306610 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:19.307269 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:19.307394 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:19.307512 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:19.308452 (+ 940us) server_negotiation.cc:299] Negotiation successful
0810 18:52:19.308560 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":156,"threads_started":1}
I20260810 18:52:19.308970 6804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:19.302209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:44587 (local address 127.0.0.1:43514)
0810 18:52:19.302424 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:19.302436 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:19.302519 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:19.303734 (+ 1215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:19.303737 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:19.303784 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:19.303901 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:19.303909 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:19.305119 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:19.305122 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:19.306188 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:19.306199 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:19.307336 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:19.307339 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:19.307447 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:19.307466 (+ 19us) client_negotiation.cc:769] Sending connection context
0810 18:52:19.307579 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 18:52:19.307652 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":67,"thread_start_us":91,"threads_started":1}
I20260810 18:52:19.311061 6713 tablet_service.cc:1467] Processing CreateTablet for tablet 9636b16653bf42d9ab3de377c8249013 (DEFAULT_TABLE table=test-table [id=f59ce32d3e9349799333af68d4753e9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:52:19.311319 6401 tablet_service.cc:1467] Processing CreateTablet for tablet 9636b16653bf42d9ab3de377c8249013 (DEFAULT_TABLE table=test-table [id=f59ce32d3e9349799333af68d4753e9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:52:19.311322 6559 tablet_service.cc:1467] Processing CreateTablet for tablet 9636b16653bf42d9ab3de377c8249013 (DEFAULT_TABLE table=test-table [id=f59ce32d3e9349799333af68d4753e9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:52:19.311944 6713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9636b16653bf42d9ab3de377c8249013. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:19.312291 6559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9636b16653bf42d9ab3de377c8249013. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:19.312290 6401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9636b16653bf42d9ab3de377c8249013. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:19.318859 6807 tablet_bootstrap.cc:492] T 9636b16653bf42d9ab3de377c8249013 P f98f188e8b5c4abfb39296e38d9dc1cb: Bootstrap starting.
I20260810 18:52:19.319437 6808 tablet_bootstrap.cc:492] T 9636b16653bf42d9ab3de377c8249013 P 6d200bb633c84796aef74f47dc576c69: Bootstrap starting.
I20260810 18:52:19.320988 6809 tablet_bootstrap.cc:492] T 9636b16653bf42d9ab3de377c8249013 P 1c27ca78b05045cbbc88e4b17860654f: Bootstrap starting.
I20260810 18:52:19.321532 6808 tablet_bootstrap.cc:654] T 9636b16653bf42d9ab3de377c8249013 P 6d200bb633c84796aef74f47dc576c69: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:19.321532 6807 tablet_bootstrap.cc:654] T 9636b16653bf42d9ab3de377c8249013 P f98f188e8b5c4abfb39296e38d9dc1cb: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:19.322476 6808 log.cc:824] T 9636b16653bf42d9ab3de377c8249013 P 6d200bb633c84796aef74f47dc576c69: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:19.322783 6807 log.cc:824] T 9636b16653bf42d9ab3de377c8249013 P f98f188e8b5c4abfb39296e38d9dc1cb: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:19.323084 6809 tablet_bootstrap.cc:654] T 9636b16653bf42d9ab3de377c8249013 P 1c27ca78b05045cbbc88e4b17860654f: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:19.324291 6809 log.cc:824] T 9636b16653bf42d9ab3de377c8249013 P 1c27ca78b05045cbbc88e4b17860654f: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:19.324684 6807 tablet_bootstrap.cc:492] T 9636b16653bf42d9ab3de377c8249013 P f98f188e8b5c4abfb39296e38d9dc1cb: No bootstrap required, opened a new log
I20260810 18:52:19.324679 6808 tablet_bootstrap.cc:492] T 9636b16653bf42d9ab3de377c8249013 P 6d200bb633c84796aef74f47dc576c69: No bootstrap required, opened a new log
I20260810 18:52:19.324899 6808 ts_tablet_manager.cc:1397] T 9636b16653bf42d9ab3de377c8249013 P 6d200bb633c84796aef74f47dc576c69: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260810 18:52:19.324899 6807 ts_tablet_manager.cc:1397] T 9636b16653bf42d9ab3de377c8249013 P f98f188e8b5c4abfb39296e38d9dc1cb: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.001s
I20260810 18:52:19.326092 6809 tablet_bootstrap.cc:492] T 9636b16653bf42d9ab3de377c8249013 P 1c27ca78b05045cbbc88e4b17860654f: No bootstrap required, opened a new log
I20260810 18:52:19.326285 6809 ts_tablet_manager.cc:1397] T 9636b16653bf42d9ab3de377c8249013 P 1c27ca78b05045cbbc88e4b17860654f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 18:52:19.329237 6809 raft_consensus.cc:348] T 9636b16653bf42d9ab3de377c8249013 P 1c27ca78b05045cbbc88e4b17860654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f98f188e8b5c4abfb39296e38d9dc1cb" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 41085 } } peers { permanent_uuid: "1c27ca78b05045cbbc88e4b17860654f" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 41551 } } peers { permanent_uuid: "6d200bb633c84796aef74f47dc576c69" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44587 } }
I20260810 18:52:19.329212 6807 raft_consensus.cc:348] T 9636b16653bf42d9ab3de377c8249013 P f98f188e8b5c4abfb39296e38d9dc1cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f98f188e8b5c4abfb39296e38d9dc1cb" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 41085 } } peers { permanent_uuid: "1c27ca78b05045cbbc88e4b17860654f" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 41551 } } peers { permanent_uuid: "6d200bb633c84796aef74f47dc576c69" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44587 } }
I20260810 18:52:19.329448 6807 raft_consensus.cc:374] T 9636b16653bf42d9ab3de377c8249013 P f98f188e8b5c4abfb39296e38d9dc1cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:19.329449 6809 raft_consensus.cc:374] T 9636b16653bf42d9ab3de377c8249013 P 1c27ca78b05045cbbc88e4b17860654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:19.329509 6807 raft_consensus.cc:729] T 9636b16653bf42d9ab3de377c8249013 P f98f188e8b5c4abfb39296e38d9dc1cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f98f188e8b5c4abfb39296e38d9dc1cb, State: Initialized, Role: FOLLOWER
I20260810 18:52:19.329509 6809 raft_consensus.cc:729] T 9636b16653bf42d9ab3de377c8249013 P 1c27ca78b05045cbbc88e4b17860654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c27ca78b05045cbbc88e4b17860654f, State: Initialized, Role: FOLLOWER
I20260810 18:52:19.329499 6808 raft_consensus.cc:348] T 9636b16653bf42d9ab3de377c8249013 P 6d200bb633c84796aef74f47dc576c69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f98f188e8b5c4abfb39296e38d9dc1cb" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 41085 } } peers { permanent_uuid: "1c27ca78b05045cbbc88e4b17860654f" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 41551 } } peers { permanent_uuid: "6d200bb633c84796aef74f47dc576c69" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44587 } }
I20260810 18:52:19.329756 6808 raft_consensus.cc:374] T 9636b16653bf42d9ab3de377c8249013 P 6d200bb633c84796aef74f47dc576c69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:19.329818 6808 raft_consensus.cc:729] T 9636b16653bf42d9ab3de377c8249013 P 6d200bb633c84796aef74f47dc576c69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d200bb633c84796aef74f47dc576c69, State: Initialized, Role: FOLLOWER
I20260810 18:52:19.329977 6809 consensus_queue.cc:260] T 9636b16653bf42d9ab3de377c8249013 P 1c27ca78b05045cbbc88e4b17860654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f98f188e8b5c4abfb39296e38d9dc1cb" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 41085 } } peers { permanent_uuid: "1c27ca78b05045cbbc88e4b17860654f" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 41551 } } peers { permanent_uuid: "6d200bb633c84796aef74f47dc576c69" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44587 } }
I20260810 18:52:19.330003 6807 consensus_queue.cc:260] T 9636b16653bf42d9ab3de377c8249013 P f98f188e8b5c4abfb39296e38d9dc1cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f98f188e8b5c4abfb39296e38d9dc1cb" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 41085 } } peers { permanent_uuid: "1c27ca78b05045cbbc88e4b17860654f" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 41551 } } peers { permanent_uuid: "6d200bb633c84796aef74f47dc576c69" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44587 } }
I20260810 18:52:19.330192 6808 consensus_queue.cc:260] T 9636b16653bf42d9ab3de377c8249013 P 6d200bb633c84796aef74f47dc576c69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f98f188e8b5c4abfb39296e38d9dc1cb" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 41085 } } peers { permanent_uuid: "1c27ca78b05045cbbc88e4b17860654f" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 41551 } } peers { permanent_uuid: "6d200bb633c84796aef74f47dc576c69" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44587 } }
I20260810 18:52:19.331019 6808 ts_tablet_manager.cc:1428] T 9636b16653bf42d9ab3de377c8249013 P 6d200bb633c84796aef74f47dc576c69: Time spent starting tablet: real 0.006s user 0.004s sys 0.002s
I20260810 18:52:19.331019 6807 ts_tablet_manager.cc:1428] T 9636b16653bf42d9ab3de377c8249013 P f98f188e8b5c4abfb39296e38d9dc1cb: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260810 18:52:19.331055 6809 ts_tablet_manager.cc:1428] T 9636b16653bf42d9ab3de377c8249013 P 1c27ca78b05045cbbc88e4b17860654f: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260810 18:52:19.331364 6787 heartbeater.cc:503] Master 127.18.168.190:39667 was elected leader, sending a full tablet report...
I20260810 18:52:19.331364 6479 heartbeater.cc:503] Master 127.18.168.190:39667 was elected leader, sending a full tablet report...
I20260810 18:52:19.331368 6634 heartbeater.cc:503] Master 127.18.168.190:39667 was elected leader, sending a full tablet report...
I20260810 18:52:19.336076 6815 raft_consensus.cc:482] T 9636b16653bf42d9ab3de377c8249013 P f98f188e8b5c4abfb39296e38d9dc1cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:52:19.336187 6815 raft_consensus.cc:504] T 9636b16653bf42d9ab3de377c8249013 P f98f188e8b5c4abfb39296e38d9dc1cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f98f188e8b5c4abfb39296e38d9dc1cb" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 41085 } } peers { permanent_uuid: "1c27ca78b05045cbbc88e4b17860654f" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 41551 } } peers { permanent_uuid: "6d200bb633c84796aef74f47dc576c69" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44587 } }
I20260810 18:52:19.337195 6815 leader_election.cc:288] T 9636b16653bf42d9ab3de377c8249013 P f98f188e8b5c4abfb39296e38d9dc1cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c27ca78b05045cbbc88e4b17860654f (127.18.168.130:41551), 6d200bb633c84796aef74f47dc576c69 (127.18.168.129:44587)
I20260810 18:52:19.341691 6805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:19.337448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:60921 (local address 127.18.168.130:41551)
0810 18:52:19.337591 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:19.337596 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:19.337609 (+ 13us) server_negotiation.cc:407] Connection header received
0810 18:52:19.337706 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:19.337709 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:19.337757 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:19.337834 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:19.338148 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:19.338933 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:19.340455 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:19.341119 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:19.341237 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:19.341307 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:19.341491 (+ 184us) server_negotiation.cc:299] Negotiation successful
0810 18:52:19.341582 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 18:52:19.341691 6789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:19.337363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:41551 (local address 127.18.168.131:60921)
0810 18:52:19.337506 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:19.337533 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:19.337619 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:19.337871 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:19.337874 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:19.337895 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:19.338036 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:19.338044 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:19.339064 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:19.339067 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:19.340330 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:19.340342 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:19.341226 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:19.341228 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:19.341337 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:19.341365 (+ 28us) client_negotiation.cc:769] Sending connection context
0810 18:52:19.341455 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 18:52:19.341534 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260810 18:52:19.341724 6806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:19.337685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:46869 (local address 127.18.168.129:44587)
0810 18:52:19.337823 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:19.337828 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:19.338059 (+ 231us) server_negotiation.cc:407] Connection header received
0810 18:52:19.338154 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:19.338156 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:19.338199 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:19.338273 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:19.338496 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:19.339201 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:19.340603 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:19.341238 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:19.341345 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:19.341384 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:19.341563 (+ 179us) server_negotiation.cc:299] Negotiation successful
0810 18:52:19.341630 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 18:52:19.341701 6819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:19.337605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:44587 (local address 127.18.168.131:46869)
0810 18:52:19.337934 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:19.337946 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:19.338071 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:19.338292 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:19.338294 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:19.338306 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:19.338398 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:19.338402 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:19.339317 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:19.339323 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:19.340486 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:19.340494 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:19.341338 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:19.341341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:19.341454 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:19.341475 (+ 21us) client_negotiation.cc:769] Sending connection context
0810 18:52:19.341537 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 18:52:19.341587 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":85,"threads_started":1}
I20260810 18:52:19.342303 6583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9636b16653bf42d9ab3de377c8249013" candidate_uuid: "f98f188e8b5c4abfb39296e38d9dc1cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c27ca78b05045cbbc88e4b17860654f" is_pre_election: true
I20260810 18:52:19.342592 6583 raft_consensus.cc:2454] T 9636b16653bf42d9ab3de377c8249013 P 1c27ca78b05045cbbc88e4b17860654f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f98f188e8b5c4abfb39296e38d9dc1cb in term 0.
I20260810 18:52:19.342599 6423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9636b16653bf42d9ab3de377c8249013" candidate_uuid: "f98f188e8b5c4abfb39296e38d9dc1cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d200bb633c84796aef74f47dc576c69" is_pre_election: true
I20260810 18:52:19.342845 6423 raft_consensus.cc:2454] T 9636b16653bf42d9ab3de377c8249013 P 6d200bb633c84796aef74f47dc576c69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f98f188e8b5c4abfb39296e38d9dc1cb in term 0.
I20260810 18:52:19.343111 6657 leader_election.cc:302] T 9636b16653bf42d9ab3de377c8249013 P f98f188e8b5c4abfb39296e38d9dc1cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c27ca78b05045cbbc88e4b17860654f, f98f188e8b5c4abfb39296e38d9dc1cb; no voters:
I20260810 18:52:19.343374 6815 raft_consensus.cc:2781] T 9636b16653bf42d9ab3de377c8249013 P f98f188e8b5c4abfb39296e38d9dc1cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:52:19.343443 6815 raft_consensus.cc:482] T 9636b16653bf42d9ab3de377c8249013 P f98f188e8b5c4abfb39296e38d9dc1cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:52:19.343479 6815 raft_consensus.cc:3037] T 9636b16653bf42d9ab3de377c8249013 P f98f188e8b5c4abfb39296e38d9dc1cb [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:19.344434 6815 raft_consensus.cc:504] T 9636b16653bf42d9ab3de377c8249013 P f98f188e8b5c4abfb39296e38d9dc1cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f98f188e8b5c4abfb39296e38d9dc1cb" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 41085 } } peers { permanent_uuid: "1c27ca78b05045cbbc88e4b17860654f" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 41551 } } peers { permanent_uuid: "6d200bb633c84796aef74f47dc576c69" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44587 } }
I20260810 18:52:19.344796 6815 leader_election.cc:288] T 9636b16653bf42d9ab3de377c8249013 P f98f188e8b5c4abfb39296e38d9dc1cb [CANDIDATE]: Term 1 election: Requested vote from peers 1c27ca78b05045cbbc88e4b17860654f (127.18.168.130:41551), 6d200bb633c84796aef74f47dc576c69 (127.18.168.129:44587)
I20260810 18:52:19.345089 6583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9636b16653bf42d9ab3de377c8249013" candidate_uuid: "f98f188e8b5c4abfb39296e38d9dc1cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c27ca78b05045cbbc88e4b17860654f"
I20260810 18:52:19.345194 6423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9636b16653bf42d9ab3de377c8249013" candidate_uuid: "f98f188e8b5c4abfb39296e38d9dc1cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d200bb633c84796aef74f47dc576c69"
I20260810 18:52:19.345222 6583 raft_consensus.cc:3037] T 9636b16653bf42d9ab3de377c8249013 P 1c27ca78b05045cbbc88e4b17860654f [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:19.345332 6423 raft_consensus.cc:3037] T 9636b16653bf42d9ab3de377c8249013 P 6d200bb633c84796aef74f47dc576c69 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:19.346530 6583 raft_consensus.cc:2454] T 9636b16653bf42d9ab3de377c8249013 P 1c27ca78b05045cbbc88e4b17860654f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f98f188e8b5c4abfb39296e38d9dc1cb in term 1.
I20260810 18:52:19.346530 6423 raft_consensus.cc:2454] T 9636b16653bf42d9ab3de377c8249013 P 6d200bb633c84796aef74f47dc576c69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f98f188e8b5c4abfb39296e38d9dc1cb in term 1.
I20260810 18:52:19.346829 6657 leader_election.cc:302] T 9636b16653bf42d9ab3de377c8249013 P f98f188e8b5c4abfb39296e38d9dc1cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c27ca78b05045cbbc88e4b17860654f, f98f188e8b5c4abfb39296e38d9dc1cb; no voters:
I20260810 18:52:19.346987 6815 raft_consensus.cc:2781] T 9636b16653bf42d9ab3de377c8249013 P f98f188e8b5c4abfb39296e38d9dc1cb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:52:19.347211 6815 raft_consensus.cc:686] T 9636b16653bf42d9ab3de377c8249013 P f98f188e8b5c4abfb39296e38d9dc1cb [term 1 LEADER]: Becoming Leader. State: Replica: f98f188e8b5c4abfb39296e38d9dc1cb, State: Running, Role: LEADER
I20260810 18:52:19.347513 6815 consensus_queue.cc:237] T 9636b16653bf42d9ab3de377c8249013 P f98f188e8b5c4abfb39296e38d9dc1cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f98f188e8b5c4abfb39296e38d9dc1cb" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 41085 } } peers { permanent_uuid: "1c27ca78b05045cbbc88e4b17860654f" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 41551 } } peers { permanent_uuid: "6d200bb633c84796aef74f47dc576c69" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44587 } }
I20260810 18:52:19.350611 6255 catalog_manager.cc:5495] T 9636b16653bf42d9ab3de377c8249013 P f98f188e8b5c4abfb39296e38d9dc1cb reported cstate change: term changed from 0 to 1, leader changed from <none> to f98f188e8b5c4abfb39296e38d9dc1cb (127.18.168.131). New cstate: current_term: 1 leader_uuid: "f98f188e8b5c4abfb39296e38d9dc1cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f98f188e8b5c4abfb39296e38d9dc1cb" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 41085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c27ca78b05045cbbc88e4b17860654f" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 41551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d200bb633c84796aef74f47dc576c69" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44587 } health_report { overall_health: UNKNOWN } } }
I20260810 18:52:19.369764 6802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:19.366466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46934 (local address 127.18.168.131:41085)
0810 18:52:19.366600 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:19.366604 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:19.366617 (+ 13us) server_negotiation.cc:407] Connection header received
0810 18:52:19.366762 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:19.366764 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:19.366816 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:19.366905 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:52:19.367117 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:19.367612 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:19.368399 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:19.368607 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:19.368679 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:19.368999 (+ 320us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:52:19.369394 (+ 395us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:52:19.369484 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:19.369596 (+ 112us) server_negotiation.cc:299] Negotiation successful
0810 18:52:19.369645 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 18:52:19.375557 6423 raft_consensus.cc:1261] T 9636b16653bf42d9ab3de377c8249013 P 6d200bb633c84796aef74f47dc576c69 [term 1 FOLLOWER]: Refusing update from remote peer f98f188e8b5c4abfb39296e38d9dc1cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:52:19.375548 6583 raft_consensus.cc:1261] T 9636b16653bf42d9ab3de377c8249013 P 1c27ca78b05045cbbc88e4b17860654f [term 1 FOLLOWER]: Refusing update from remote peer f98f188e8b5c4abfb39296e38d9dc1cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:52:19.376305 6820 consensus_queue.cc:1035] T 9636b16653bf42d9ab3de377c8249013 P f98f188e8b5c4abfb39296e38d9dc1cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c27ca78b05045cbbc88e4b17860654f" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 41551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:52:19.376515 6815 consensus_queue.cc:1035] T 9636b16653bf42d9ab3de377c8249013 P f98f188e8b5c4abfb39296e38d9dc1cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d200bb633c84796aef74f47dc576c69" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:52:19.383592 6824 mvcc.cc:205] Tried to move back new op lower bound from 7317044999674732544 to 7317044999569784832. Current Snapshot: MvccSnapshot[applied={T|T < 7317044999674732544}]
I20260810 18:52:19.385870 6825 mvcc.cc:205] Tried to move back new op lower bound from 7317044999674732544 to 7317044999569784832. Current Snapshot: MvccSnapshot[applied={T|T < 7317044999674732544}]
I20260810 18:52:19.386065 6826 mvcc.cc:205] Tried to move back new op lower bound from 7317044999674732544 to 7317044999569784832. Current Snapshot: MvccSnapshot[applied={T|T < 7317044999674732544}]
I20260810 18:52:19.391368 6255 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48504:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:52:19.391536 6255 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48504:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:52:19.393976 6255 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c77c3890120546419e596619784ad997: Sending DeleteTablet for 3 replicas of tablet 9636b16653bf42d9ab3de377c8249013
I20260810 18:52:19.394685 6713 tablet_service.cc:1514] Processing DeleteTablet for tablet 9636b16653bf42d9ab3de377c8249013 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:52:19 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:46926
I20260810 18:52:19.394730 6401 tablet_service.cc:1514] Processing DeleteTablet for tablet 9636b16653bf42d9ab3de377c8249013 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:52:19 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:43514
I20260810 18:52:19.395174 6837 tablet_replica.cc:330] T 9636b16653bf42d9ab3de377c8249013 P 6d200bb633c84796aef74f47dc576c69: stopping tablet replica
I20260810 18:52:19.395155 6559 tablet_service.cc:1514] Processing DeleteTablet for tablet 9636b16653bf42d9ab3de377c8249013 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:52:19 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:51680
I20260810 18:52:19.395510 6837 raft_consensus.cc:2229] T 9636b16653bf42d9ab3de377c8249013 P 6d200bb633c84796aef74f47dc576c69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:52:19.395581 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 6327
I20260810 18:52:19.400924 6836 tablet_replica.cc:330] T 9636b16653bf42d9ab3de377c8249013 P f98f188e8b5c4abfb39296e38d9dc1cb: stopping tablet replica
I20260810 18:52:19.401206 6836 raft_consensus.cc:2229] T 9636b16653bf42d9ab3de377c8249013 P f98f188e8b5c4abfb39296e38d9dc1cb [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:52:19.401583 6836 raft_consensus.cc:2258] T 9636b16653bf42d9ab3de377c8249013 P f98f188e8b5c4abfb39296e38d9dc1cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:52:19.401921 6838 tablet_replica.cc:330] T 9636b16653bf42d9ab3de377c8249013 P 1c27ca78b05045cbbc88e4b17860654f: stopping tablet replica
I20260810 18:52:19.402213 6838 raft_consensus.cc:2229] T 9636b16653bf42d9ab3de377c8249013 P 1c27ca78b05045cbbc88e4b17860654f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:52:19.402489 6838 raft_consensus.cc:2258] T 9636b16653bf42d9ab3de377c8249013 P 1c27ca78b05045cbbc88e4b17860654f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:52:19.404639 6838 ts_tablet_manager.cc:1905] T 9636b16653bf42d9ab3de377c8249013 P 1c27ca78b05045cbbc88e4b17860654f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:52:19.407833 6838 ts_tablet_manager.cc:1918] T 9636b16653bf42d9ab3de377c8249013 P 1c27ca78b05045cbbc88e4b17860654f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:52:19.407919 6838 log.cc:1196] T 9636b16653bf42d9ab3de377c8249013 P 1c27ca78b05045cbbc88e4b17860654f: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-1/wal/wals/9636b16653bf42d9ab3de377c8249013
I20260810 18:52:19.408257 6838 ts_tablet_manager.cc:1939] T 9636b16653bf42d9ab3de377c8249013 P 1c27ca78b05045cbbc88e4b17860654f: Deleting consensus metadata
I20260810 18:52:19.409456 6238 catalog_manager.cc:4841] TS 1c27ca78b05045cbbc88e4b17860654f (127.18.168.130:41551): tablet 9636b16653bf42d9ab3de377c8249013 (table test-table [id=f59ce32d3e9349799333af68d4753e9f]) successfully deleted
I20260810 18:52:19.410792 6836 ts_tablet_manager.cc:1905] T 9636b16653bf42d9ab3de377c8249013 P f98f188e8b5c4abfb39296e38d9dc1cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:52:19.413301 6836 ts_tablet_manager.cc:1918] T 9636b16653bf42d9ab3de377c8249013 P f98f188e8b5c4abfb39296e38d9dc1cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:52:19.413390 6836 log.cc:1196] T 9636b16653bf42d9ab3de377c8249013 P f98f188e8b5c4abfb39296e38d9dc1cb: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387687857264-19106-0/minicluster-data/ts-2/wal/wals/9636b16653bf42d9ab3de377c8249013
I20260810 18:52:19.413735 6836 ts_tablet_manager.cc:1939] T 9636b16653bf42d9ab3de377c8249013 P f98f188e8b5c4abfb39296e38d9dc1cb: Deleting consensus metadata
I20260810 18:52:19.414847 6241 catalog_manager.cc:4841] TS f98f188e8b5c4abfb39296e38d9dc1cb (127.18.168.131:41085): tablet 9636b16653bf42d9ab3de377c8249013 (table test-table [id=f59ce32d3e9349799333af68d4753e9f]) successfully deleted
W20260810 18:52:19.420687 6238 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.168.129:44587 (error 108)
I20260810 18:52:19.420878 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 6484
I20260810 18:52:19.422533 6804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:19.422169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:44587 (local address 127.0.0.1:43528)
0810 18:52:19.422356 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:19.422432 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.129:44587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
W20260810 18:52:19.422755 6238 catalog_manager.cc:4584] TS 6d200bb633c84796aef74f47dc576c69 (127.18.168.129:44587): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9636b16653bf42d9ab3de377c8249013: Network error: Client connection negotiation failed: client connection to 127.18.168.129:44587: connect: Connection refused (error 111)
I20260810 18:52:19.437495 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 6639
I20260810 18:52:19.445907 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 6217
2026-08-10T18:52:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2354 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-10T18:52:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:52:19Z Disabled control of system clock
I20260810 18:52:19.480108 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:37311
--webserver_interface=127.18.168.190
--webserver_port=0
--builtin_ntp_servers=127.18.168.148:43075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:37311
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 18:52:19.589581 6844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:19.589823 6844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:19.589859 6844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:19.593701 6844 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:52:19.593745 6844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:19.593760 6844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:19.593775 6844 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:52:19.593789 6844 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:52:19.598543 6844 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:43075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:37311
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:37311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:19.599521 6844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:19.600798 6844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:19.609207 6849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:19.609213 6850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:19.609172 6853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:19.609481 6844 server_base.cc:1034] running on GCE node
I20260810 18:52:19.610006 6844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:19.611030 6844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:19.612221 6844 hybrid_clock.cc:648] HybridClock initialized: now 1786387939612200 us; error 29 us; skew 500 ppm
I20260810 18:52:19.614065 6844 webserver.cc:460] Webserver started at http://127.18.168.190:42791/ using document root <none> and password file <none>
I20260810 18:52:19.614615 6844 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:19.614707 6844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:19.614933 6844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:19.616993 6844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "0c9f4a2f912d459c89e9a7b2fe01d350"
format_stamp: "Formatted at 2026-08-10 18:52:19 on dist-test-slave-cbsf"
server_key: "dcc7c1b52fa974294113b2e178f32832"
server_key_iv: "f4376accc31c8fe3bcd3a0ec3c0a4263"
server_key_version: "encryptionkey@0"
I20260810 18:52:19.617579 6844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "0c9f4a2f912d459c89e9a7b2fe01d350"
format_stamp: "Formatted at 2026-08-10 18:52:19 on dist-test-slave-cbsf"
server_key: "dcc7c1b52fa974294113b2e178f32832"
server_key_iv: "f4376accc31c8fe3bcd3a0ec3c0a4263"
server_key_version: "encryptionkey@0"
I20260810 18:52:19.621218 6844 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260810 18:52:19.623594 6860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:19.624903 6844 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 18:52:19.625044 6844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "0c9f4a2f912d459c89e9a7b2fe01d350"
format_stamp: "Formatted at 2026-08-10 18:52:19 on dist-test-slave-cbsf"
server_key: "dcc7c1b52fa974294113b2e178f32832"
server_key_iv: "f4376accc31c8fe3bcd3a0ec3c0a4263"
server_key_version: "encryptionkey@0"
I20260810 18:52:19.625149 6844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:19.647435 6844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:19.648089 6844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:19.648274 6844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:19.655514 6844 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:37311
I20260810 18:52:19.655566 6919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:37311 every 8 connection(s)
I20260810 18:52:19.656512 6844 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:52:19.659567 6920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:19.665083 6920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c9f4a2f912d459c89e9a7b2fe01d350: Bootstrap starting.
I20260810 18:52:19.666082 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 6844
I20260810 18:52:19.666218 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:52:19.666553 19106 external_mini_cluster.cc:1428] Setting key f6edeb9f05835e036b3998cb52d90218
I20260810 18:52:19.668154 6920 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c9f4a2f912d459c89e9a7b2fe01d350: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:19.668977 6920 log.cc:824] T 00000000000000000000000000000000 P 0c9f4a2f912d459c89e9a7b2fe01d350: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:19.670811 6920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c9f4a2f912d459c89e9a7b2fe01d350: No bootstrap required, opened a new log
I20260810 18:52:19.674253 6920 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c9f4a2f912d459c89e9a7b2fe01d350 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c9f4a2f912d459c89e9a7b2fe01d350" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 37311 } }
I20260810 18:52:19.674450 6920 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c9f4a2f912d459c89e9a7b2fe01d350 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:19.674510 6920 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c9f4a2f912d459c89e9a7b2fe01d350 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c9f4a2f912d459c89e9a7b2fe01d350, State: Initialized, Role: FOLLOWER
I20260810 18:52:19.675009 6920 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c9f4a2f912d459c89e9a7b2fe01d350 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c9f4a2f912d459c89e9a7b2fe01d350" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 37311 } }
I20260810 18:52:19.675166 6920 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c9f4a2f912d459c89e9a7b2fe01d350 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:52:19.675251 6920 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c9f4a2f912d459c89e9a7b2fe01d350 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:52:19.675362 6920 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c9f4a2f912d459c89e9a7b2fe01d350 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:19.675937 6924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:19.667691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54444 (local address 127.18.168.190:37311)
0810 18:52:19.668392 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:19.668409 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:19.668442 (+ 33us) server_negotiation.cc:407] Connection header received
0810 18:52:19.669179 (+ 737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:19.669198 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:19.669585 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:19.669907 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:19.670334 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:19.671396 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:19.672392 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:19.672935 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:19.673107 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:19.673991 (+ 884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:19.673996 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:19.674009 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:52:19.674380 (+ 371us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:19.674457 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:52:19.674487 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:19.674606 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:19.674774 (+ 168us) server_negotiation.cc:299] Negotiation successful
0810 18:52:19.675025 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":137,"threads_started":1}
I20260810 18:52:19.676335 6920 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c9f4a2f912d459c89e9a7b2fe01d350 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c9f4a2f912d459c89e9a7b2fe01d350" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 37311 } }
I20260810 18:52:19.676666 6920 leader_election.cc:302] T 00000000000000000000000000000000 P 0c9f4a2f912d459c89e9a7b2fe01d350 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c9f4a2f912d459c89e9a7b2fe01d350; no voters:
I20260810 18:52:19.677058 6920 leader_election.cc:288] T 00000000000000000000000000000000 P 0c9f4a2f912d459c89e9a7b2fe01d350 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:52:19.677170 6926 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c9f4a2f912d459c89e9a7b2fe01d350 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:52:19.677418 6926 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c9f4a2f912d459c89e9a7b2fe01d350 [term 1 LEADER]: Becoming Leader. State: Replica: 0c9f4a2f912d459c89e9a7b2fe01d350, State: Running, Role: LEADER
I20260810 18:52:19.677724 6926 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c9f4a2f912d459c89e9a7b2fe01d350 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c9f4a2f912d459c89e9a7b2fe01d350" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 37311 } }
I20260810 18:52:19.678334 6920 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c9f4a2f912d459c89e9a7b2fe01d350 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:52:19.678855 6928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c9f4a2f912d459c89e9a7b2fe01d350 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c9f4a2f912d459c89e9a7b2fe01d350. Latest consensus state: current_term: 1 leader_uuid: "0c9f4a2f912d459c89e9a7b2fe01d350" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c9f4a2f912d459c89e9a7b2fe01d350" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 37311 } } }
I20260810 18:52:19.679000 6928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c9f4a2f912d459c89e9a7b2fe01d350 [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:19.679268 6927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c9f4a2f912d459c89e9a7b2fe01d350 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c9f4a2f912d459c89e9a7b2fe01d350" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c9f4a2f912d459c89e9a7b2fe01d350" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 37311 } } }
I20260810 18:52:19.679397 6927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c9f4a2f912d459c89e9a7b2fe01d350 [sys.catalog]: This master's current role is: LEADER
W20260810 18:52:19.682835 6943 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0c9f4a2f912d459c89e9a7b2fe01d350: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:52:19.682901 6943 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:52:19.682983 6932 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:52:19.683825 6932 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:52:19.689141 6932 catalog_manager.cc:1347] Generated new cluster ID: bda78d24ad574b539bd2a54a3454af13
I20260810 18:52:19.689210 6932 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:52:19.702709 6932 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:52:19.703594 6932 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:52:19.728168 6932 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c9f4a2f912d459c89e9a7b2fe01d350: Generated new TSK 0
I20260810 18:52:19.729127 6932 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:52:19.739199 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.129:0
--local_ip_for_outbound_sockets=127.18.168.129
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:37311
--builtin_ntp_servers=127.18.168.148:43075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 18:52:19.873615 6947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:19.873862 6947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:19.873900 6947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:19.877851 6947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:19.877899 6947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:19.877951 6947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.129
I20260810 18:52:19.882524 6947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:43075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:37311
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.129
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:19.883538 6947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:19.884884 6947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:19.894711 6956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:19.894727 6959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:19.894711 6957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:19.895021 6947 server_base.cc:1034] running on GCE node
I20260810 18:52:19.895536 6947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:19.896176 6947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:19.897374 6947 hybrid_clock.cc:648] HybridClock initialized: now 1786387939897352 us; error 38 us; skew 500 ppm
I20260810 18:52:19.899261 6947 webserver.cc:460] Webserver started at http://127.18.168.129:42049/ using document root <none> and password file <none>
I20260810 18:52:19.900028 6947 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:19.900117 6947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:19.900365 6947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:19.902663 6947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-0/data/instance:
uuid: "a6de2f51c7c94a60b64bea690fb3a71a"
format_stamp: "Formatted at 2026-08-10 18:52:19 on dist-test-slave-cbsf"
server_key: "7e0572d4e88f6e6a46dcf69b9b0cd654"
server_key_iv: "c81a8c2b4f2d2fa4860cf3e7830a6698"
server_key_version: "encryptionkey@0"
I20260810 18:52:19.903264 6947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance:
uuid: "a6de2f51c7c94a60b64bea690fb3a71a"
format_stamp: "Formatted at 2026-08-10 18:52:19 on dist-test-slave-cbsf"
server_key: "7e0572d4e88f6e6a46dcf69b9b0cd654"
server_key_iv: "c81a8c2b4f2d2fa4860cf3e7830a6698"
server_key_version: "encryptionkey@0"
I20260810 18:52:19.907333 6947 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:52:19.909749 6965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:19.910928 6947 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:52:19.911088 6947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-0/wal
uuid: "a6de2f51c7c94a60b64bea690fb3a71a"
format_stamp: "Formatted at 2026-08-10 18:52:19 on dist-test-slave-cbsf"
server_key: "7e0572d4e88f6e6a46dcf69b9b0cd654"
server_key_iv: "c81a8c2b4f2d2fa4860cf3e7830a6698"
server_key_version: "encryptionkey@0"
I20260810 18:52:19.911202 6947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:19.922734 6947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:19.923409 6947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:19.923614 6947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:19.924206 6947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:52:19.925203 6947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:52:19.925273 6947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:19.925339 6947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:52:19.925392 6947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:19.934675 6947 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.129:38161
I20260810 18:52:19.934724 7097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.129:38161 every 8 connection(s)
I20260810 18:52:19.935554 6947 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
I20260810 18:52:19.937625 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 6947
I20260810 18:52:19.937758 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance
I20260810 18:52:19.938048 19106 external_mini_cluster.cc:1428] Setting key 542f58fec2a544406cf6dcb1b126fc7e
I20260810 18:52:19.940021 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.130:0
--local_ip_for_outbound_sockets=127.18.168.130
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:37311
--builtin_ntp_servers=127.18.168.148:43075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 18:52:19.946053 6924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:19.937327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:48817 (local address 127.18.168.190:37311)
0810 18:52:19.937498 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:19.937503 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:19.938668 (+ 1165us) server_negotiation.cc:407] Connection header received
0810 18:52:19.940062 (+ 1394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:19.940066 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:19.940127 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:19.940218 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:19.941180 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:19.942031 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:19.943706 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:19.944148 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:19.944323 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:19.945418 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:19.945425 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:19.945431 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:52:19.945451 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:19.945480 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:52:19.945495 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:19.945599 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:19.945748 (+ 149us) server_negotiation.cc:299] Negotiation successful
0810 18:52:19.945835 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 18:52:19.946849 7100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:19.937761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:37311 (local address 127.18.168.129:48817)
0810 18:52:19.938509 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:19.938562 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:19.939845 (+ 1283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:19.940417 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:19.940423 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:19.940503 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:52:19.941033 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:19.941054 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:19.942163 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:19.942170 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:19.943551 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:19.943562 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:19.944334 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:19.944337 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:19.944477 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:19.945143 (+ 666us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:52:19.945144 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:52:19.945183 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:52:19.945246 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:52:19.945596 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:52:19.945604 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:52:19.945608 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 18:52:19.945709 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 18:52:19.945920 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":124,"threads_started":1}
I20260810 18:52:19.948181 7098 heartbeater.cc:347] Connected to a master server at 127.18.168.190:37311
I20260810 18:52:19.948436 7098 heartbeater.cc:464] Registering TS with master...
I20260810 18:52:19.948992 7098 heartbeater.cc:511] Master 127.18.168.190:37311 requested a full tablet report, sending...
I20260810 18:52:19.950519 6880 ts_manager.cc:194] Registered new tserver with Master: a6de2f51c7c94a60b64bea690fb3a71a (127.18.168.129:38161)
I20260810 18:52:19.951697 6880 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.168.129:48817
W20260810 18:52:20.058966 7101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:20.059221 7101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:20.059258 7101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:20.063357 7101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:20.063416 7101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:20.063506 7101 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.130
I20260810 18:52:20.068234 7101 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:43075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:37311
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.130
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:20.069444 7101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:20.070848 7101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:20.081004 7110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:20.081142 7101 server_base.cc:1034] running on GCE node
W20260810 18:52:20.080976 7107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:20.081020 7106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:20.081679 7101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:20.082365 7101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:20.083571 7101 hybrid_clock.cc:648] HybridClock initialized: now 1786387940083548 us; error 35 us; skew 500 ppm
I20260810 18:52:20.085682 7101 webserver.cc:460] Webserver started at http://127.18.168.130:37523/ using document root <none> and password file <none>
I20260810 18:52:20.086300 7101 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:20.086395 7101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:20.086671 7101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:20.089001 7101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-1/data/instance:
uuid: "33c50467f8504720ab8a14b428d5a147"
format_stamp: "Formatted at 2026-08-10 18:52:20 on dist-test-slave-cbsf"
server_key: "b6ef9beeb742ed6156f4865290c7c7a9"
server_key_iv: "cf945078fc40402e25987748a3ee7a0c"
server_key_version: "encryptionkey@0"
I20260810 18:52:20.089608 7101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance:
uuid: "33c50467f8504720ab8a14b428d5a147"
format_stamp: "Formatted at 2026-08-10 18:52:20 on dist-test-slave-cbsf"
server_key: "b6ef9beeb742ed6156f4865290c7c7a9"
server_key_iv: "cf945078fc40402e25987748a3ee7a0c"
server_key_version: "encryptionkey@0"
I20260810 18:52:20.093485 7101 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 18:52:20.095992 7119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:20.097275 7101 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:52:20.097417 7101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-1/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-1/wal
uuid: "33c50467f8504720ab8a14b428d5a147"
format_stamp: "Formatted at 2026-08-10 18:52:20 on dist-test-slave-cbsf"
server_key: "b6ef9beeb742ed6156f4865290c7c7a9"
server_key_iv: "cf945078fc40402e25987748a3ee7a0c"
server_key_version: "encryptionkey@0"
I20260810 18:52:20.097543 7101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:20.122396 7101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:20.123205 7101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:20.123443 7101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:20.124099 7101 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:52:20.125213 7101 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:52:20.125290 7101 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:20.125371 7101 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:52:20.125424 7101 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:20.135269 7101 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.130:43583
I20260810 18:52:20.135291 7243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.130:43583 every 8 connection(s)
I20260810 18:52:20.136230 7101 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
I20260810 18:52:20.138801 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 7101
I20260810 18:52:20.138908 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance
I20260810 18:52:20.139312 19106 external_mini_cluster.cc:1428] Setting key 9cc5b1c49d68c74b7cdeac78baeded83
I20260810 18:52:20.141721 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.131:0
--local_ip_for_outbound_sockets=127.18.168.131
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:37311
--builtin_ntp_servers=127.18.168.148:43075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 18:52:20.146818 6924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:20.138265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:46943 (local address 127.18.168.190:37311)
0810 18:52:20.138408 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:20.138412 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:20.139777 (+ 1365us) server_negotiation.cc:407] Connection header received
0810 18:52:20.141391 (+ 1614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:20.141394 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:20.141438 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:20.141514 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:20.142799 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:20.143357 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:20.144710 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:20.145031 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:20.145130 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:20.146294 (+ 1164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:20.146297 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:20.146300 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:52:20.146313 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:20.146331 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:52:20.146339 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:20.146421 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:20.146595 (+ 174us) server_negotiation.cc:299] Negotiation successful
0810 18:52:20.146662 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 18:52:20.147748 7247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:20.138708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:37311 (local address 127.18.168.130:46943)
0810 18:52:20.139567 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:20.139627 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:20.141140 (+ 1513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:20.141735 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:20.141747 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:20.141866 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:52:20.142623 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:20.142645 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:20.143560 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:20.143566 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:20.144548 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:20.144558 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:20.145185 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:20.145188 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:20.145337 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:20.146017 (+ 680us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:52:20.146019 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:52:20.146060 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:52:20.146126 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:52:20.146423 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:52:20.146436 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:52:20.146444 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 18:52:20.146583 (+ 139us) client_negotiation.cc:240] Negotiation successful
0810 18:52:20.146879 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":160,"threads_started":1}
I20260810 18:52:20.149133 7244 heartbeater.cc:347] Connected to a master server at 127.18.168.190:37311
I20260810 18:52:20.149389 7244 heartbeater.cc:464] Registering TS with master...
I20260810 18:52:20.149961 7244 heartbeater.cc:511] Master 127.18.168.190:37311 requested a full tablet report, sending...
I20260810 18:52:20.151194 6880 ts_manager.cc:194] Registered new tserver with Master: 33c50467f8504720ab8a14b428d5a147 (127.18.168.130:43583)
I20260810 18:52:20.152101 6880 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.168.130:46943
W20260810 18:52:20.260850 7248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:20.261088 7248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:20.261126 7248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:20.264969 7248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:20.265012 7248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:20.265064 7248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.131
I20260810 18:52:20.269713 7248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:43075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:37311
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.131
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:20.270730 7248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:20.272061 7248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:20.281803 7257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:20.281803 7254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:20.281805 7255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:20.282016 7248 server_base.cc:1034] running on GCE node
I20260810 18:52:20.282593 7248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:20.283238 7248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:20.284490 7248 hybrid_clock.cc:648] HybridClock initialized: now 1786387940284454 us; error 45 us; skew 500 ppm
I20260810 18:52:20.286388 7248 webserver.cc:460] Webserver started at http://127.18.168.131:46297/ using document root <none> and password file <none>
I20260810 18:52:20.287040 7248 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:20.287137 7248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:20.287376 7248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:20.289588 7248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-2/data/instance:
uuid: "8ade9f9ed3494c0a8a40b4e9862100ef"
format_stamp: "Formatted at 2026-08-10 18:52:20 on dist-test-slave-cbsf"
server_key: "cd8217b8d4169164aefcaea52d37c764"
server_key_iv: "d111451818a5effb0bb0070216a84282"
server_key_version: "encryptionkey@0"
I20260810 18:52:20.290207 7248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance:
uuid: "8ade9f9ed3494c0a8a40b4e9862100ef"
format_stamp: "Formatted at 2026-08-10 18:52:20 on dist-test-slave-cbsf"
server_key: "cd8217b8d4169164aefcaea52d37c764"
server_key_iv: "d111451818a5effb0bb0070216a84282"
server_key_version: "encryptionkey@0"
I20260810 18:52:20.294139 7248 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:52:20.296607 7264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:20.297869 7248 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:52:20.298024 7248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-2/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-2/wal
uuid: "8ade9f9ed3494c0a8a40b4e9862100ef"
format_stamp: "Formatted at 2026-08-10 18:52:20 on dist-test-slave-cbsf"
server_key: "cd8217b8d4169164aefcaea52d37c764"
server_key_iv: "d111451818a5effb0bb0070216a84282"
server_key_version: "encryptionkey@0"
I20260810 18:52:20.298138 7248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:20.325431 7248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:20.326181 7248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:20.326409 7248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:20.327039 7248 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:52:20.328078 7248 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:52:20.328153 7248 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:20.328231 7248 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:52:20.328275 7248 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:20.338060 7248 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.131:40239
I20260810 18:52:20.338120 7394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.131:40239 every 8 connection(s)
I20260810 18:52:20.338997 7248 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
I20260810 18:52:20.340118 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 7248
I20260810 18:52:20.340214 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance
I20260810 18:52:20.340631 19106 external_mini_cluster.cc:1428] Setting key e7a83d92fe3cbb4e84d6848f071ded4e
I20260810 18:52:20.348840 6924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:20.341064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:60663 (local address 127.18.168.190:37311)
0810 18:52:20.341239 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:20.341245 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:20.342550 (+ 1305us) server_negotiation.cc:407] Connection header received
0810 18:52:20.344047 (+ 1497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:20.344050 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:20.344099 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:20.344191 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:20.345123 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:20.345732 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:20.346872 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:20.347149 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:20.347243 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:20.348349 (+ 1106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:20.348352 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:20.348355 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:52:20.348367 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:20.348387 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:52:20.348396 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:20.348501 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:20.348642 (+ 141us) server_negotiation.cc:299] Negotiation successful
0810 18:52:20.348726 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 18:52:20.349681 7397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:20.341529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:37311 (local address 127.18.168.131:60663)
0810 18:52:20.342392 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:20.342447 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:20.343836 (+ 1389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:20.344315 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:20.344321 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:20.344399 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:52:20.344942 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:20.344986 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:20.345861 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:20.345864 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:20.346752 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:20.346762 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:20.347310 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:20.347314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:20.347470 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:20.348111 (+ 641us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:52:20.348112 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:52:20.348150 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:52:20.348209 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:52:20.348513 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:52:20.348521 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:52:20.348525 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 18:52:20.348617 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 18:52:20.348876 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":173,"threads_started":1}
I20260810 18:52:20.350796 7395 heartbeater.cc:347] Connected to a master server at 127.18.168.190:37311
I20260810 18:52:20.351022 7395 heartbeater.cc:464] Registering TS with master...
I20260810 18:52:20.351519 7395 heartbeater.cc:511] Master 127.18.168.190:37311 requested a full tablet report, sending...
I20260810 18:52:20.352577 6880 ts_manager.cc:194] Registered new tserver with Master: 8ade9f9ed3494c0a8a40b4e9862100ef (127.18.168.131:40239)
I20260810 18:52:20.353176 6880 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.168.131:60663
I20260810 18:52:20.354807 19106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:52:20.360455 6924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:20.356786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42020 (local address 127.18.168.190:37311)
0810 18:52:20.356875 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:20.356879 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:20.357118 (+ 239us) server_negotiation.cc:407] Connection header received
0810 18:52:20.357239 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:20.357241 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:20.357281 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:20.357349 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:20.357567 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:20.358030 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:20.358940 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:20.359145 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:20.359227 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:20.360030 (+ 803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:20.360033 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:20.360045 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:52:20.360056 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:20.360073 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:52:20.360081 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:20.360137 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:20.360242 (+ 105us) server_negotiation.cc:299] Negotiation successful
0810 18:52:20.360287 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 18:52:20.363255 6880 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42020:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:52:20.366119 6880 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:52:20.381662 7410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:20.375672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:43583 (local address 127.0.0.1:55484)
0810 18:52:20.376223 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:20.376236 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:20.376319 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:20.377120 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:20.377124 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:20.377144 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:20.377260 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:20.377266 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:20.378800 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:20.378804 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:20.379981 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:20.379991 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:20.381029 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:20.381033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:20.381165 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:20.381212 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 18:52:20.381338 (+ 126us) client_negotiation.cc:240] Negotiation successful
0810 18:52:20.381492 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"mutex_wait_us":89,"thread_start_us":95,"threads_started":1}
I20260810 18:52:20.381675 7412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:20.375917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55484 (local address 127.18.168.130:43583)
0810 18:52:20.376627 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:20.376641 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:20.376667 (+ 26us) server_negotiation.cc:407] Connection header received
0810 18:52:20.376764 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:20.376769 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:20.376962 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:20.377409 (+ 447us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:20.377561 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:20.378668 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:20.380107 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:20.380885 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:20.381009 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:20.381195 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:20.381400 (+ 205us) server_negotiation.cc:299] Negotiation successful
0810 18:52:20.381502 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":115,"threads_started":1}
I20260810 18:52:20.382138 7406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:20.374900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.131:40239 (local address 127.0.0.1:40316)
0810 18:52:20.375528 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:20.375582 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:20.375725 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:20.377049 (+ 1324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:20.377052 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:20.377070 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:20.377200 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:20.377207 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:20.378389 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:20.378395 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:20.380319 (+ 1924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:20.380332 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:20.381672 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:20.381676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:20.381816 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:20.381842 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 18:52:20.381916 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 18:52:20.381989 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"mutex_wait_us":188,"thread_start_us":92,"threads_started":1}
I20260810 18:52:20.382521 7409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:20.375625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45394 (local address 127.18.168.129:38161)
0810 18:52:20.376070 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:20.376086 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:20.376305 (+ 219us) server_negotiation.cc:407] Connection header received
0810 18:52:20.376405 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:20.376411 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:20.376656 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:20.376855 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:20.377180 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:20.378407 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:20.380488 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:20.381384 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:20.381506 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:20.381681 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:20.382229 (+ 548us) server_negotiation.cc:299] Negotiation successful
0810 18:52:20.382343 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":126,"threads_started":1}
I20260810 18:52:20.382722 7408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:20.375311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40316 (local address 127.18.168.131:40239)
0810 18:52:20.376546 (+ 1235us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:20.376557 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:20.376581 (+ 24us) server_negotiation.cc:407] Connection header received
0810 18:52:20.376657 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:20.376662 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:20.376878 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:20.377036 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:20.377325 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:20.378264 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:20.380457 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:20.381298 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:20.381433 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:20.381603 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:20.382432 (+ 829us) server_negotiation.cc:299] Negotiation successful
0810 18:52:20.382530 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1127,"thread_start_us":185,"threads_started":1}
I20260810 18:52:20.382941 7407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:20.375096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:38161 (local address 127.0.0.1:45394)
0810 18:52:20.376133 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:20.376149 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:20.376253 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:20.376848 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:20.376854 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:20.376876 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:20.377009 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:20.377021 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:20.378524 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:20.378527 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:20.380319 (+ 1792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:20.380332 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:20.381760 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:20.381767 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:20.381915 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:20.381965 (+ 50us) client_negotiation.cc:769] Sending connection context
0810 18:52:20.382039 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 18:52:20.382105 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":966,"mutex_wait_us":104,"thread_start_us":116,"threads_started":1}
I20260810 18:52:20.385509 7170 tablet_service.cc:1467] Processing CreateTablet for tablet 5e46580194d9408689d617d88bc0941c (DEFAULT_TABLE table=test-table [id=a2bd6c40dee646f1b1eaa0adfd38e0f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:52:20.385509 7320 tablet_service.cc:1467] Processing CreateTablet for tablet 5e46580194d9408689d617d88bc0941c (DEFAULT_TABLE table=test-table [id=a2bd6c40dee646f1b1eaa0adfd38e0f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:52:20.386670 7320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e46580194d9408689d617d88bc0941c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:20.386670 7170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e46580194d9408689d617d88bc0941c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:20.387079 7024 tablet_service.cc:1467] Processing CreateTablet for tablet 5e46580194d9408689d617d88bc0941c (DEFAULT_TABLE table=test-table [id=a2bd6c40dee646f1b1eaa0adfd38e0f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:52:20.388049 7024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e46580194d9408689d617d88bc0941c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:20.394147 7413 tablet_bootstrap.cc:492] T 5e46580194d9408689d617d88bc0941c P a6de2f51c7c94a60b64bea690fb3a71a: Bootstrap starting.
I20260810 18:52:20.395387 7414 tablet_bootstrap.cc:492] T 5e46580194d9408689d617d88bc0941c P 33c50467f8504720ab8a14b428d5a147: Bootstrap starting.
I20260810 18:52:20.396992 7413 tablet_bootstrap.cc:654] T 5e46580194d9408689d617d88bc0941c P a6de2f51c7c94a60b64bea690fb3a71a: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:20.396993 7415 tablet_bootstrap.cc:492] T 5e46580194d9408689d617d88bc0941c P 8ade9f9ed3494c0a8a40b4e9862100ef: Bootstrap starting.
I20260810 18:52:20.397667 7414 tablet_bootstrap.cc:654] T 5e46580194d9408689d617d88bc0941c P 33c50467f8504720ab8a14b428d5a147: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:20.397957 7413 log.cc:824] T 5e46580194d9408689d617d88bc0941c P a6de2f51c7c94a60b64bea690fb3a71a: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:20.398543 7414 log.cc:824] T 5e46580194d9408689d617d88bc0941c P 33c50467f8504720ab8a14b428d5a147: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:20.399075 7415 tablet_bootstrap.cc:654] T 5e46580194d9408689d617d88bc0941c P 8ade9f9ed3494c0a8a40b4e9862100ef: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:20.399854 7415 log.cc:824] T 5e46580194d9408689d617d88bc0941c P 8ade9f9ed3494c0a8a40b4e9862100ef: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:20.400282 7413 tablet_bootstrap.cc:492] T 5e46580194d9408689d617d88bc0941c P a6de2f51c7c94a60b64bea690fb3a71a: No bootstrap required, opened a new log
I20260810 18:52:20.400511 7413 ts_tablet_manager.cc:1397] T 5e46580194d9408689d617d88bc0941c P a6de2f51c7c94a60b64bea690fb3a71a: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260810 18:52:20.400588 7414 tablet_bootstrap.cc:492] T 5e46580194d9408689d617d88bc0941c P 33c50467f8504720ab8a14b428d5a147: No bootstrap required, opened a new log
I20260810 18:52:20.400815 7414 ts_tablet_manager.cc:1397] T 5e46580194d9408689d617d88bc0941c P 33c50467f8504720ab8a14b428d5a147: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260810 18:52:20.401991 7415 tablet_bootstrap.cc:492] T 5e46580194d9408689d617d88bc0941c P 8ade9f9ed3494c0a8a40b4e9862100ef: No bootstrap required, opened a new log
I20260810 18:52:20.402195 7415 ts_tablet_manager.cc:1397] T 5e46580194d9408689d617d88bc0941c P 8ade9f9ed3494c0a8a40b4e9862100ef: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260810 18:52:20.405172 7413 raft_consensus.cc:348] T 5e46580194d9408689d617d88bc0941c P a6de2f51c7c94a60b64bea690fb3a71a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ade9f9ed3494c0a8a40b4e9862100ef" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 40239 } } peers { permanent_uuid: "a6de2f51c7c94a60b64bea690fb3a71a" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 38161 } } peers { permanent_uuid: "33c50467f8504720ab8a14b428d5a147" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 43583 } }
I20260810 18:52:20.405133 7414 raft_consensus.cc:348] T 5e46580194d9408689d617d88bc0941c P 33c50467f8504720ab8a14b428d5a147 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ade9f9ed3494c0a8a40b4e9862100ef" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 40239 } } peers { permanent_uuid: "a6de2f51c7c94a60b64bea690fb3a71a" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 38161 } } peers { permanent_uuid: "33c50467f8504720ab8a14b428d5a147" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 43583 } }
I20260810 18:52:20.405439 7413 raft_consensus.cc:374] T 5e46580194d9408689d617d88bc0941c P a6de2f51c7c94a60b64bea690fb3a71a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:20.405457 7414 raft_consensus.cc:374] T 5e46580194d9408689d617d88bc0941c P 33c50467f8504720ab8a14b428d5a147 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:20.405496 7413 raft_consensus.cc:729] T 5e46580194d9408689d617d88bc0941c P a6de2f51c7c94a60b64bea690fb3a71a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6de2f51c7c94a60b64bea690fb3a71a, State: Initialized, Role: FOLLOWER
I20260810 18:52:20.405515 7414 raft_consensus.cc:729] T 5e46580194d9408689d617d88bc0941c P 33c50467f8504720ab8a14b428d5a147 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33c50467f8504720ab8a14b428d5a147, State: Initialized, Role: FOLLOWER
I20260810 18:52:20.405982 7414 consensus_queue.cc:260] T 5e46580194d9408689d617d88bc0941c P 33c50467f8504720ab8a14b428d5a147 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ade9f9ed3494c0a8a40b4e9862100ef" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 40239 } } peers { permanent_uuid: "a6de2f51c7c94a60b64bea690fb3a71a" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 38161 } } peers { permanent_uuid: "33c50467f8504720ab8a14b428d5a147" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 43583 } }
I20260810 18:52:20.405966 7413 consensus_queue.cc:260] T 5e46580194d9408689d617d88bc0941c P a6de2f51c7c94a60b64bea690fb3a71a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ade9f9ed3494c0a8a40b4e9862100ef" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 40239 } } peers { permanent_uuid: "a6de2f51c7c94a60b64bea690fb3a71a" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 38161 } } peers { permanent_uuid: "33c50467f8504720ab8a14b428d5a147" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 43583 } }
I20260810 18:52:20.406716 7098 heartbeater.cc:503] Master 127.18.168.190:37311 was elected leader, sending a full tablet report...
I20260810 18:52:20.407027 7414 ts_tablet_manager.cc:1428] T 5e46580194d9408689d617d88bc0941c P 33c50467f8504720ab8a14b428d5a147: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260810 18:52:20.407178 7413 ts_tablet_manager.cc:1428] T 5e46580194d9408689d617d88bc0941c P a6de2f51c7c94a60b64bea690fb3a71a: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260810 18:52:20.407436 7244 heartbeater.cc:503] Master 127.18.168.190:37311 was elected leader, sending a full tablet report...
I20260810 18:52:20.407666 7415 raft_consensus.cc:348] T 5e46580194d9408689d617d88bc0941c P 8ade9f9ed3494c0a8a40b4e9862100ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ade9f9ed3494c0a8a40b4e9862100ef" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 40239 } } peers { permanent_uuid: "a6de2f51c7c94a60b64bea690fb3a71a" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 38161 } } peers { permanent_uuid: "33c50467f8504720ab8a14b428d5a147" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 43583 } }
I20260810 18:52:20.407919 7415 raft_consensus.cc:374] T 5e46580194d9408689d617d88bc0941c P 8ade9f9ed3494c0a8a40b4e9862100ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:20.407987 7415 raft_consensus.cc:729] T 5e46580194d9408689d617d88bc0941c P 8ade9f9ed3494c0a8a40b4e9862100ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ade9f9ed3494c0a8a40b4e9862100ef, State: Initialized, Role: FOLLOWER
I20260810 18:52:20.408422 7415 consensus_queue.cc:260] T 5e46580194d9408689d617d88bc0941c P 8ade9f9ed3494c0a8a40b4e9862100ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ade9f9ed3494c0a8a40b4e9862100ef" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 40239 } } peers { permanent_uuid: "a6de2f51c7c94a60b64bea690fb3a71a" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 38161 } } peers { permanent_uuid: "33c50467f8504720ab8a14b428d5a147" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 43583 } }
I20260810 18:52:20.409145 7395 heartbeater.cc:503] Master 127.18.168.190:37311 was elected leader, sending a full tablet report...
I20260810 18:52:20.409371 7415 ts_tablet_manager.cc:1428] T 5e46580194d9408689d617d88bc0941c P 8ade9f9ed3494c0a8a40b4e9862100ef: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
W20260810 18:52:20.436774 7099 tablet.cc:2369] T 5e46580194d9408689d617d88bc0941c P a6de2f51c7c94a60b64bea690fb3a71a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:52:20.590063 7396 tablet.cc:2369] T 5e46580194d9408689d617d88bc0941c P 8ade9f9ed3494c0a8a40b4e9862100ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:52:20.637544 7246 tablet.cc:2369] T 5e46580194d9408689d617d88bc0941c P 33c50467f8504720ab8a14b428d5a147: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:52:20.713569 7422 raft_consensus.cc:482] T 5e46580194d9408689d617d88bc0941c P 8ade9f9ed3494c0a8a40b4e9862100ef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:52:20.713840 7422 raft_consensus.cc:504] T 5e46580194d9408689d617d88bc0941c P 8ade9f9ed3494c0a8a40b4e9862100ef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ade9f9ed3494c0a8a40b4e9862100ef" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 40239 } } peers { permanent_uuid: "a6de2f51c7c94a60b64bea690fb3a71a" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 38161 } } peers { permanent_uuid: "33c50467f8504720ab8a14b428d5a147" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 43583 } }
I20260810 18:52:20.715044 7422 leader_election.cc:288] T 5e46580194d9408689d617d88bc0941c P 8ade9f9ed3494c0a8a40b4e9862100ef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6de2f51c7c94a60b64bea690fb3a71a (127.18.168.129:38161), 33c50467f8504720ab8a14b428d5a147 (127.18.168.130:43583)
I20260810 18:52:20.719709 7409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:20.715481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:38895 (local address 127.18.168.129:38161)
0810 18:52:20.715632 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:20.715639 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:20.715657 (+ 18us) server_negotiation.cc:407] Connection header received
0810 18:52:20.715726 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:20.715731 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:20.715805 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:20.715880 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:20.716218 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:20.717147 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:20.718462 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:20.719059 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:20.719179 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:20.719233 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:20.719527 (+ 294us) server_negotiation.cc:299] Negotiation successful
0810 18:52:20.719603 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 18:52:20.719710 7397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:20.715298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:38161 (local address 127.18.168.131:38895)
0810 18:52:20.715433 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:20.715450 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:20.715563 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:20.715904 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:20.715907 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:20.715924 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:20.716064 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:20.716073 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:20.717303 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:20.717307 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:20.718290 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:20.718301 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:20.719208 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:20.719212 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:20.719366 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:20.719394 (+ 28us) client_negotiation.cc:769] Sending connection context
0810 18:52:20.719495 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 18:52:20.719571 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260810 18:52:20.719972 7420 raft_consensus.cc:482] T 5e46580194d9408689d617d88bc0941c P a6de2f51c7c94a60b64bea690fb3a71a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:52:20.720177 7420 raft_consensus.cc:504] T 5e46580194d9408689d617d88bc0941c P a6de2f51c7c94a60b64bea690fb3a71a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ade9f9ed3494c0a8a40b4e9862100ef" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 40239 } } peers { permanent_uuid: "a6de2f51c7c94a60b64bea690fb3a71a" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 38161 } } peers { permanent_uuid: "33c50467f8504720ab8a14b428d5a147" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 43583 } }
I20260810 18:52:20.720330 7424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:20.715298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:43583 (local address 127.18.168.131:45627)
0810 18:52:20.716158 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:20.716173 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:20.716253 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:20.716533 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:20.716543 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:20.716564 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:20.716747 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:20.716754 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:20.717614 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:20.717617 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:20.718670 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:20.718686 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:20.719903 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:20.719908 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:20.720033 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:20.720057 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 18:52:20.720125 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 18:52:20.720200 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":783,"mutex_wait_us":98,"thread_start_us":119,"threads_started":1}
I20260810 18:52:20.720448 7045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e46580194d9408689d617d88bc0941c" candidate_uuid: "8ade9f9ed3494c0a8a40b4e9862100ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6de2f51c7c94a60b64bea690fb3a71a" is_pre_election: true
I20260810 18:52:20.720719 7045 raft_consensus.cc:2454] T 5e46580194d9408689d617d88bc0941c P a6de2f51c7c94a60b64bea690fb3a71a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ade9f9ed3494c0a8a40b4e9862100ef in term 0.
I20260810 18:52:20.721236 7266 leader_election.cc:302] T 5e46580194d9408689d617d88bc0941c P 8ade9f9ed3494c0a8a40b4e9862100ef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ade9f9ed3494c0a8a40b4e9862100ef, a6de2f51c7c94a60b64bea690fb3a71a; no voters:
I20260810 18:52:20.721541 7420 leader_election.cc:288] T 5e46580194d9408689d617d88bc0941c P a6de2f51c7c94a60b64bea690fb3a71a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ade9f9ed3494c0a8a40b4e9862100ef (127.18.168.131:40239), 33c50467f8504720ab8a14b428d5a147 (127.18.168.130:43583)
I20260810 18:52:20.722661 7422 raft_consensus.cc:2781] T 5e46580194d9408689d617d88bc0941c P 8ade9f9ed3494c0a8a40b4e9862100ef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:52:20.722771 7422 raft_consensus.cc:482] T 5e46580194d9408689d617d88bc0941c P 8ade9f9ed3494c0a8a40b4e9862100ef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:52:20.722827 7422 raft_consensus.cc:3037] T 5e46580194d9408689d617d88bc0941c P 8ade9f9ed3494c0a8a40b4e9862100ef [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:20.724339 7422 raft_consensus.cc:504] T 5e46580194d9408689d617d88bc0941c P 8ade9f9ed3494c0a8a40b4e9862100ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ade9f9ed3494c0a8a40b4e9862100ef" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 40239 } } peers { permanent_uuid: "a6de2f51c7c94a60b64bea690fb3a71a" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 38161 } } peers { permanent_uuid: "33c50467f8504720ab8a14b428d5a147" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 43583 } }
I20260810 18:52:20.724908 7422 leader_election.cc:288] T 5e46580194d9408689d617d88bc0941c P 8ade9f9ed3494c0a8a40b4e9862100ef [CANDIDATE]: Term 1 election: Requested vote from peers a6de2f51c7c94a60b64bea690fb3a71a (127.18.168.129:38161), 33c50467f8504720ab8a14b428d5a147 (127.18.168.130:43583)
I20260810 18:52:20.725026 7045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e46580194d9408689d617d88bc0941c" candidate_uuid: "8ade9f9ed3494c0a8a40b4e9862100ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6de2f51c7c94a60b64bea690fb3a71a"
I20260810 18:52:20.725157 7045 raft_consensus.cc:3037] T 5e46580194d9408689d617d88bc0941c P a6de2f51c7c94a60b64bea690fb3a71a [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:20.725299 7412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:20.715813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:45627 (local address 127.18.168.130:43583)
0810 18:52:20.716267 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:20.716274 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:20.716291 (+ 17us) server_negotiation.cc:407] Connection header received
0810 18:52:20.716347 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:20.716350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:20.716408 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:20.716486 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:20.716898 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:20.717505 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:20.718801 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:20.719408 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:20.719543 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:20.719608 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:20.725053 (+ 5445us) server_negotiation.cc:299] Negotiation successful
0810 18:52:20.725158 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362}
I20260810 18:52:20.726074 7195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e46580194d9408689d617d88bc0941c" candidate_uuid: "8ade9f9ed3494c0a8a40b4e9862100ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33c50467f8504720ab8a14b428d5a147" is_pre_election: true
I20260810 18:52:20.726378 7195 raft_consensus.cc:2454] T 5e46580194d9408689d617d88bc0941c P 33c50467f8504720ab8a14b428d5a147 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ade9f9ed3494c0a8a40b4e9862100ef in term 0.
I20260810 18:52:20.726408 7045 raft_consensus.cc:2454] T 5e46580194d9408689d617d88bc0941c P a6de2f51c7c94a60b64bea690fb3a71a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ade9f9ed3494c0a8a40b4e9862100ef in term 1.
I20260810 18:52:20.726740 7193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e46580194d9408689d617d88bc0941c" candidate_uuid: "8ade9f9ed3494c0a8a40b4e9862100ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33c50467f8504720ab8a14b428d5a147"
I20260810 18:52:20.726825 7266 leader_election.cc:302] T 5e46580194d9408689d617d88bc0941c P 8ade9f9ed3494c0a8a40b4e9862100ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ade9f9ed3494c0a8a40b4e9862100ef, a6de2f51c7c94a60b64bea690fb3a71a; no voters:
I20260810 18:52:20.726871 7193 raft_consensus.cc:3037] T 5e46580194d9408689d617d88bc0941c P 33c50467f8504720ab8a14b428d5a147 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:20.726962 7425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:20.721820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:43583 (local address 127.18.168.129:51123)
0810 18:52:20.722042 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:20.722057 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:20.722143 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:20.722978 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:20.722981 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:20.722997 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:20.723145 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:20.723153 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:20.723866 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:20.723871 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:20.725380 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:20.725393 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:20.726449 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:20.726453 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:20.726674 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:20.726698 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 18:52:20.726758 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 18:52:20.726823 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":92,"threads_started":1}
I20260810 18:52:20.727715 7427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:20.722300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:51123 (local address 127.18.168.130:43583)
0810 18:52:20.722647 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:20.722651 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:20.722671 (+ 20us) server_negotiation.cc:407] Connection header received
0810 18:52:20.722729 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:20.722731 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:20.722779 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:20.722865 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:20.723254 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:20.723755 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:20.725546 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:20.726168 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:20.726301 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:20.726368 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:20.727426 (+ 1058us) server_negotiation.cc:299] Negotiation successful
0810 18:52:20.727572 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":90,"threads_started":1}
I20260810 18:52:20.728256 7193 raft_consensus.cc:2454] T 5e46580194d9408689d617d88bc0941c P 33c50467f8504720ab8a14b428d5a147 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ade9f9ed3494c0a8a40b4e9862100ef in term 1.
I20260810 18:52:20.728856 7193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e46580194d9408689d617d88bc0941c" candidate_uuid: "a6de2f51c7c94a60b64bea690fb3a71a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33c50467f8504720ab8a14b428d5a147" is_pre_election: true
I20260810 18:52:20.729025 7193 raft_consensus.cc:2379] T 5e46580194d9408689d617d88bc0941c P 33c50467f8504720ab8a14b428d5a147 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a6de2f51c7c94a60b64bea690fb3a71a in current term 1: Already voted for candidate 8ade9f9ed3494c0a8a40b4e9862100ef in this term.
I20260810 18:52:20.729395 7422 raft_consensus.cc:2781] T 5e46580194d9408689d617d88bc0941c P 8ade9f9ed3494c0a8a40b4e9862100ef [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:52:20.729696 7422 raft_consensus.cc:686] T 5e46580194d9408689d617d88bc0941c P 8ade9f9ed3494c0a8a40b4e9862100ef [term 1 LEADER]: Becoming Leader. State: Replica: 8ade9f9ed3494c0a8a40b4e9862100ef, State: Running, Role: LEADER
I20260810 18:52:20.730047 7422 consensus_queue.cc:237] T 5e46580194d9408689d617d88bc0941c P 8ade9f9ed3494c0a8a40b4e9862100ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ade9f9ed3494c0a8a40b4e9862100ef" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 40239 } } peers { permanent_uuid: "a6de2f51c7c94a60b64bea690fb3a71a" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 38161 } } peers { permanent_uuid: "33c50467f8504720ab8a14b428d5a147" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 43583 } }
I20260810 18:52:20.731009 7408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:20.722564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:43721 (local address 127.18.168.131:40239)
0810 18:52:20.722668 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:20.722674 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:20.727282 (+ 4608us) server_negotiation.cc:407] Connection header received
0810 18:52:20.727388 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:20.727391 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:20.727436 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:20.727516 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:20.727727 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:20.728390 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:20.729707 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:20.730279 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:20.730399 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:20.730473 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:20.730821 (+ 348us) server_negotiation.cc:299] Negotiation successful
0810 18:52:20.730891 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 18:52:20.731035 7425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:20.721876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.131:40239 (local address 127.18.168.129:43721)
0810 18:52:20.727206 (+ 5330us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:20.727223 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:20.727311 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:20.727538 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:20.727541 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:20.727557 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:20.727632 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:20.727637 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:20.728500 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:20.728504 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:20.729554 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:20.729568 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:20.730581 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:20.730585 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:20.730704 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:20.730727 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 18:52:20.730803 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 18:52:20.730871 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5265,"thread_start_us":100,"threads_started":1}
I20260810 18:52:20.731596 7343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e46580194d9408689d617d88bc0941c" candidate_uuid: "a6de2f51c7c94a60b64bea690fb3a71a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ade9f9ed3494c0a8a40b4e9862100ef" is_pre_election: true
I20260810 18:52:20.732188 6972 leader_election.cc:302] T 5e46580194d9408689d617d88bc0941c P a6de2f51c7c94a60b64bea690fb3a71a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6de2f51c7c94a60b64bea690fb3a71a; no voters: 33c50467f8504720ab8a14b428d5a147, 8ade9f9ed3494c0a8a40b4e9862100ef
I20260810 18:52:20.732715 7420 raft_consensus.cc:2726] T 5e46580194d9408689d617d88bc0941c P a6de2f51c7c94a60b64bea690fb3a71a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 18:52:20.734077 6879 catalog_manager.cc:5495] T 5e46580194d9408689d617d88bc0941c P 8ade9f9ed3494c0a8a40b4e9862100ef reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ade9f9ed3494c0a8a40b4e9862100ef (127.18.168.131). New cstate: current_term: 1 leader_uuid: "8ade9f9ed3494c0a8a40b4e9862100ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ade9f9ed3494c0a8a40b4e9862100ef" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 40239 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6de2f51c7c94a60b64bea690fb3a71a" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 38161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33c50467f8504720ab8a14b428d5a147" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 43583 } health_report { overall_health: UNKNOWN } } }
I20260810 18:52:20.752063 7408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:20.748130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40322 (local address 127.18.168.131:40239)
0810 18:52:20.748257 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:20.748261 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:20.748387 (+ 126us) server_negotiation.cc:407] Connection header received
0810 18:52:20.748593 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:20.748601 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:20.748657 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:20.748888 (+ 231us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:52:20.749146 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:20.749656 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:20.750588 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:20.750825 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:20.750931 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:20.751262 (+ 331us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:52:20.751674 (+ 412us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:52:20.751805 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:20.751864 (+ 59us) server_negotiation.cc:299] Negotiation successful
0810 18:52:20.751915 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 18:52:20.758342 7193 raft_consensus.cc:1261] T 5e46580194d9408689d617d88bc0941c P 33c50467f8504720ab8a14b428d5a147 [term 1 FOLLOWER]: Refusing update from remote peer 8ade9f9ed3494c0a8a40b4e9862100ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:52:20.758410 7045 raft_consensus.cc:1261] T 5e46580194d9408689d617d88bc0941c P a6de2f51c7c94a60b64bea690fb3a71a [term 1 FOLLOWER]: Refusing update from remote peer 8ade9f9ed3494c0a8a40b4e9862100ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:52:20.759068 7429 consensus_queue.cc:1035] T 5e46580194d9408689d617d88bc0941c P 8ade9f9ed3494c0a8a40b4e9862100ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "33c50467f8504720ab8a14b428d5a147" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 43583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:52:20.759244 7422 consensus_queue.cc:1035] T 5e46580194d9408689d617d88bc0941c P 8ade9f9ed3494c0a8a40b4e9862100ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6de2f51c7c94a60b64bea690fb3a71a" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 38161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:52:20.767532 7435 mvcc.cc:205] Tried to move back new op lower bound from 7317045005338402816 to 7317045005232996352. Current Snapshot: MvccSnapshot[applied={T|T < 7317045005338402816}]
I20260810 18:52:20.778851 6878 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:52:20.779124 6878 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:52:20.781848 6878 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0c9f4a2f912d459c89e9a7b2fe01d350: Sending DeleteTablet for 3 replicas of tablet 5e46580194d9408689d617d88bc0941c
I20260810 18:52:20.782683 7320 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e46580194d9408689d617d88bc0941c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:52:20 UTC) from {username='slave'} at 127.0.0.1:40316
I20260810 18:52:20.783006 7170 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e46580194d9408689d617d88bc0941c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:52:20 UTC) from {username='slave'} at 127.0.0.1:55484
I20260810 18:52:20.783245 7024 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e46580194d9408689d617d88bc0941c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:52:20 UTC) from {username='slave'} at 127.0.0.1:45394
I20260810 18:52:20.783334 7442 tablet_replica.cc:330] T 5e46580194d9408689d617d88bc0941c P 8ade9f9ed3494c0a8a40b4e9862100ef: stopping tablet replica
I20260810 18:52:20.783727 7444 tablet_replica.cc:330] T 5e46580194d9408689d617d88bc0941c P a6de2f51c7c94a60b64bea690fb3a71a: stopping tablet replica
I20260810 18:52:20.783764 7442 raft_consensus.cc:2229] T 5e46580194d9408689d617d88bc0941c P 8ade9f9ed3494c0a8a40b4e9862100ef [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:52:20.784037 7444 raft_consensus.cc:2229] T 5e46580194d9408689d617d88bc0941c P a6de2f51c7c94a60b64bea690fb3a71a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:52:20.784086 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 6947
I20260810 18:52:20.784384 7444 raft_consensus.cc:2258] T 5e46580194d9408689d617d88bc0941c P a6de2f51c7c94a60b64bea690fb3a71a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:52:20.784483 7442 raft_consensus.cc:2258] T 5e46580194d9408689d617d88bc0941c P 8ade9f9ed3494c0a8a40b4e9862100ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:52:20.786872 7442 ts_tablet_manager.cc:1905] T 5e46580194d9408689d617d88bc0941c P 8ade9f9ed3494c0a8a40b4e9862100ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:52:20.790616 7442 ts_tablet_manager.cc:1918] T 5e46580194d9408689d617d88bc0941c P 8ade9f9ed3494c0a8a40b4e9862100ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:52:20.790722 7442 log.cc:1196] T 5e46580194d9408689d617d88bc0941c P 8ade9f9ed3494c0a8a40b4e9862100ef: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-2/wal/wals/5e46580194d9408689d617d88bc0941c
I20260810 18:52:20.791093 7442 ts_tablet_manager.cc:1939] T 5e46580194d9408689d617d88bc0941c P 8ade9f9ed3494c0a8a40b4e9862100ef: Deleting consensus metadata
I20260810 18:52:20.791411 7443 tablet_replica.cc:330] T 5e46580194d9408689d617d88bc0941c P 33c50467f8504720ab8a14b428d5a147: stopping tablet replica
I20260810 18:52:20.792074 7443 raft_consensus.cc:2229] T 5e46580194d9408689d617d88bc0941c P 33c50467f8504720ab8a14b428d5a147 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:52:20.792358 6865 catalog_manager.cc:4841] TS 8ade9f9ed3494c0a8a40b4e9862100ef (127.18.168.131:40239): tablet 5e46580194d9408689d617d88bc0941c (table test-table [id=a2bd6c40dee646f1b1eaa0adfd38e0f4]) successfully deleted
I20260810 18:52:20.792450 7443 raft_consensus.cc:2258] T 5e46580194d9408689d617d88bc0941c P 33c50467f8504720ab8a14b428d5a147 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:52:20.794123 7443 ts_tablet_manager.cc:1905] T 5e46580194d9408689d617d88bc0941c P 33c50467f8504720ab8a14b428d5a147: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:52:20.796797 7443 ts_tablet_manager.cc:1918] T 5e46580194d9408689d617d88bc0941c P 33c50467f8504720ab8a14b428d5a147: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:52:20.796895 7443 log.cc:1196] T 5e46580194d9408689d617d88bc0941c P 33c50467f8504720ab8a14b428d5a147: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387687857264-19106-0/minicluster-data/ts-1/wal/wals/5e46580194d9408689d617d88bc0941c
I20260810 18:52:20.797230 7443 ts_tablet_manager.cc:1939] T 5e46580194d9408689d617d88bc0941c P 33c50467f8504720ab8a14b428d5a147: Deleting consensus metadata
W20260810 18:52:20.798147 6862 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.168.129:38161 (error 108)
I20260810 18:52:20.798417 6863 catalog_manager.cc:4841] TS 33c50467f8504720ab8a14b428d5a147 (127.18.168.130:43583): tablet 5e46580194d9408689d617d88bc0941c (table test-table [id=a2bd6c40dee646f1b1eaa0adfd38e0f4]) successfully deleted
I20260810 18:52:20.799074 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 7101
I20260810 18:52:20.799367 7406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:20.799009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:38161 (local address 127.0.0.1:45402)
0810 18:52:20.799178 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:20.799264 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.129:38161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260810 18:52:20.800388 6862 catalog_manager.cc:4584] TS a6de2f51c7c94a60b64bea690fb3a71a (127.18.168.129:38161): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5e46580194d9408689d617d88bc0941c: Network error: Client connection negotiation failed: client connection to 127.18.168.129:38161: connect: Connection refused (error 111)
I20260810 18:52:20.814272 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 7248
I20260810 18:52:20.823873 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 6844
2026-08-10T18:52:20Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1376 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:52:20 dist-test-slave-cbsf krb5kdc[7449](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:52:20 dist-test-slave-cbsf krb5kdc[7449](info): set up 2 sockets
Aug 10 18:52:20 dist-test-slave-cbsf krb5kdc[7449](info): commencing operation
krb5kdc: starting...
W20260810 18:52:21.937495 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:52:22 dist-test-slave-cbsf krb5kdc[7449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:52:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:52:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.168.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
I20260810 18:52:22.071971 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:46049
--webserver_interface=127.18.168.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:44641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:46049
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:52:22.185848 7463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:22.186107 7463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:22.186148 7463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:22.190040 7463 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:52:22.190090 7463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:22.190109 7463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:22.190126 7463 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:52:22.190141 7463 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:52:22.195209 7463 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:44641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:46049
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:46049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7463
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:22.196168 7463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:22.197141 7463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:22.205616 7468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:22.205721 7463 server_base.cc:1034] running on GCE node
W20260810 18:52:22.205658 7469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:22.205667 7471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:22.206597 7463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:22.207775 7463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:22.208958 7463 hybrid_clock.cc:648] HybridClock initialized: now 1786387942208927 us; error 43 us; skew 500 ppm
Aug 10 18:52:22 dist-test-slave-cbsf krb5kdc[7449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387942, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:22.215622 7463 init.cc:377] Logged in from keytab as kudu/127.18.168.190@KRBTEST.COM (short username kudu)
I20260810 18:52:22.216738 7463 webserver.cc:460] Webserver started at http://127.18.168.190:42897/ using document root <none> and password file <none>
I20260810 18:52:22.217410 7463 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:22.217535 7463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:22.217808 7463 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:22.220186 7463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "8555d45ad5c04fecad35555aeb003541"
format_stamp: "Formatted at 2026-08-10 18:52:22 on dist-test-slave-cbsf"
server_key: "605ed9bd8d101767b7e30eaf423f758a"
server_key_iv: "b0bce8e2e05a394473a22fb4806b0b71"
server_key_version: "encryptionkey@0"
I20260810 18:52:22.220912 7463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "8555d45ad5c04fecad35555aeb003541"
format_stamp: "Formatted at 2026-08-10 18:52:22 on dist-test-slave-cbsf"
server_key: "605ed9bd8d101767b7e30eaf423f758a"
server_key_iv: "b0bce8e2e05a394473a22fb4806b0b71"
server_key_version: "encryptionkey@0"
I20260810 18:52:22.225061 7463 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:52:22.227595 7478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:22.228950 7463 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:52:22.229101 7463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "8555d45ad5c04fecad35555aeb003541"
format_stamp: "Formatted at 2026-08-10 18:52:22 on dist-test-slave-cbsf"
server_key: "605ed9bd8d101767b7e30eaf423f758a"
server_key_iv: "b0bce8e2e05a394473a22fb4806b0b71"
server_key_version: "encryptionkey@0"
I20260810 18:52:22.229214 7463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:22.243206 7463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:22.246080 7463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:22.246274 7463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:22.253428 7463 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:46049
I20260810 18:52:22.253440 7540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:46049 every 8 connection(s)
I20260810 18:52:22.254380 7463 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:52:22.257216 7541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:22.259752 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 7463
I20260810 18:52:22.259904 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:52:22.260205 19106 external_mini_cluster.cc:1428] Setting key 4a74f397a73a3d4d9dc9248568155fa0
I20260810 18:52:22.263178 7541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8555d45ad5c04fecad35555aeb003541: Bootstrap starting.
I20260810 18:52:22.265872 7541 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8555d45ad5c04fecad35555aeb003541: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:22.266782 7541 log.cc:824] T 00000000000000000000000000000000 P 8555d45ad5c04fecad35555aeb003541: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:22.269207 7541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8555d45ad5c04fecad35555aeb003541: No bootstrap required, opened a new log
Aug 10 18:52:22 dist-test-slave-cbsf krb5kdc[7449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:52:22.273450 7541 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8555d45ad5c04fecad35555aeb003541 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8555d45ad5c04fecad35555aeb003541" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 46049 } }
I20260810 18:52:22.273743 7541 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8555d45ad5c04fecad35555aeb003541 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:22.273805 7541 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8555d45ad5c04fecad35555aeb003541 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8555d45ad5c04fecad35555aeb003541, State: Initialized, Role: FOLLOWER
I20260810 18:52:22.274297 7541 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8555d45ad5c04fecad35555aeb003541 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8555d45ad5c04fecad35555aeb003541" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 46049 } }
I20260810 18:52:22.274454 7541 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8555d45ad5c04fecad35555aeb003541 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:52:22.274523 7541 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8555d45ad5c04fecad35555aeb003541 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:52:22.274619 7541 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8555d45ad5c04fecad35555aeb003541 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:22.275704 7541 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8555d45ad5c04fecad35555aeb003541 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8555d45ad5c04fecad35555aeb003541" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 46049 } }
I20260810 18:52:22.276067 7541 leader_election.cc:302] T 00000000000000000000000000000000 P 8555d45ad5c04fecad35555aeb003541 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8555d45ad5c04fecad35555aeb003541; no voters:
I20260810 18:52:22.276401 7541 leader_election.cc:288] T 00000000000000000000000000000000 P 8555d45ad5c04fecad35555aeb003541 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:52:22.276497 7547 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8555d45ad5c04fecad35555aeb003541 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:52:22.276723 7547 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8555d45ad5c04fecad35555aeb003541 [term 1 LEADER]: Becoming Leader. State: Replica: 8555d45ad5c04fecad35555aeb003541, State: Running, Role: LEADER
I20260810 18:52:22.276801 7545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:22.261730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39962 (local address 127.18.168.190:46049)
0810 18:52:22.262163 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:22.262179 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:22.262214 (+ 35us) server_negotiation.cc:407] Connection header received
0810 18:52:22.262853 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:22.262871 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:22.263225 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:22.263551 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:22.264500 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:22.265574 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:22.266810 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:22.267338 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:22.267755 (+ 417us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:22.270567 (+ 2812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:22.270573 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:22.270592 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:22.270623 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:22.273413 (+ 2790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:22.273912 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:22.273917 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:22.273922 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:22.274035 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:22.274347 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:22.274350 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:22.274351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:22.274968 (+ 617us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:22.275201 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:22.275513 (+ 312us) server_negotiation.cc:299] Negotiation successful
0810 18:52:22.275819 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":149,"threads_started":1}
I20260810 18:52:22.277061 7547 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8555d45ad5c04fecad35555aeb003541 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8555d45ad5c04fecad35555aeb003541" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 46049 } }
I20260810 18:52:22.277762 7541 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8555d45ad5c04fecad35555aeb003541 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:52:22.279678 7549 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8555d45ad5c04fecad35555aeb003541 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8555d45ad5c04fecad35555aeb003541. Latest consensus state: current_term: 1 leader_uuid: "8555d45ad5c04fecad35555aeb003541" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8555d45ad5c04fecad35555aeb003541" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 46049 } } }
I20260810 18:52:22.279816 7549 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8555d45ad5c04fecad35555aeb003541 [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:22.280202 7563 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:52:22.280159 7548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8555d45ad5c04fecad35555aeb003541 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8555d45ad5c04fecad35555aeb003541" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8555d45ad5c04fecad35555aeb003541" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 46049 } } }
I20260810 18:52:22.280623 7548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8555d45ad5c04fecad35555aeb003541 [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:22.283047 7563 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:52:22.288107 7563 catalog_manager.cc:1347] Generated new cluster ID: 8f12261b103e46b5b6b434baba517847
I20260810 18:52:22.288172 7563 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:52:22.299064 7563 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:52:22.299924 7563 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:52:22.309255 7563 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8555d45ad5c04fecad35555aeb003541: Generated new TSK 0
I20260810 18:52:22.310051 7563 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.168.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab.
Entry for principal kudu/127.18.168.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab.
I20260810 18:52:22.340646 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.129:0
--local_ip_for_outbound_sockets=127.18.168.129
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:46049
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:44641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:52:22.455974 7569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:22.456291 7569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:22.456337 7569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:22.460358 7569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:22.460412 7569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:22.460515 7569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.129
I20260810 18:52:22.465795 7569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:44641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:46049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.129
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:22.466883 7569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:22.467911 7569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:22.477396 7574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:22.477409 7575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:22.477577 7577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:22.477909 7569 server_base.cc:1034] running on GCE node
I20260810 18:52:22.478338 7569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:22.478933 7569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:22.480159 7569 hybrid_clock.cc:648] HybridClock initialized: now 1786387942480140 us; error 35 us; skew 500 ppm
Aug 10 18:52:22 dist-test-slave-cbsf krb5kdc[7449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387942, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:22.483088 7569 init.cc:377] Logged in from keytab as kudu/127.18.168.129@KRBTEST.COM (short username kudu)
I20260810 18:52:22.483924 7569 webserver.cc:460] Webserver started at http://127.18.168.129:43303/ using document root <none> and password file <none>
I20260810 18:52:22.484498 7569 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:22.484587 7569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:22.484854 7569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:22.486931 7569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-0/data/instance:
uuid: "88c77ad0619140a0afd9170c17ccb63b"
format_stamp: "Formatted at 2026-08-10 18:52:22 on dist-test-slave-cbsf"
server_key: "6121f3dd8abd1e9dffee5e12dd5b928b"
server_key_iv: "b6d3f9dce2af29dfd102ca2c94851d60"
server_key_version: "encryptionkey@0"
I20260810 18:52:22.487519 7569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance:
uuid: "88c77ad0619140a0afd9170c17ccb63b"
format_stamp: "Formatted at 2026-08-10 18:52:22 on dist-test-slave-cbsf"
server_key: "6121f3dd8abd1e9dffee5e12dd5b928b"
server_key_iv: "b6d3f9dce2af29dfd102ca2c94851d60"
server_key_version: "encryptionkey@0"
I20260810 18:52:22.491235 7569 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 18:52:22.493690 7586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:22.494886 7569 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:52:22.495020 7569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-0/wal
uuid: "88c77ad0619140a0afd9170c17ccb63b"
format_stamp: "Formatted at 2026-08-10 18:52:22 on dist-test-slave-cbsf"
server_key: "6121f3dd8abd1e9dffee5e12dd5b928b"
server_key_iv: "b6d3f9dce2af29dfd102ca2c94851d60"
server_key_version: "encryptionkey@0"
I20260810 18:52:22.495138 7569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:22.505985 7569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:22.509199 7569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:22.509377 7569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:22.509934 7569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:52:22.510844 7569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:52:22.510910 7569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:22.510975 7569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:52:22.511021 7569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:22.520226 7569 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.129:43437
I20260810 18:52:22.520248 7718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.129:43437 every 8 connection(s)
I20260810 18:52:22.521214 7569 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
I20260810 18:52:22.527472 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 7569
I20260810 18:52:22.527666 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance
I20260810 18:52:22.527989 19106 external_mini_cluster.cc:1428] Setting key 4b0bd9f7a09734b7d5c47438f771b8a1
Aug 10 18:52:22 dist-test-slave-cbsf krb5kdc[7449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387942, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:52:22.535825 7545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:22.523017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:55621 (local address 127.18.168.190:46049)
0810 18:52:22.523170 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:22.523175 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:22.524037 (+ 862us) server_negotiation.cc:407] Connection header received
0810 18:52:22.525492 (+ 1455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:22.525495 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:22.525550 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:22.525632 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:22.526839 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:22.527378 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:22.528962 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:22.529191 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:22.529321 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:22.531998 (+ 2677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:22.532001 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:22.532003 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:22.532027 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:22.533621 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:22.534227 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:22.534230 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:22.534231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:22.534303 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:22.534744 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:22.534746 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:22.534747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:22.534915 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:22.535023 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:22.535515 (+ 492us) server_negotiation.cc:299] Negotiation successful
0810 18:52:22.535651 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 18:52:22.536602 7721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:22.523382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:46049 (local address 127.18.168.129:55621)
0810 18:52:22.523866 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:22.523919 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:22.525294 (+ 1375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:22.525809 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:22.525815 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:22.526172 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:52:22.526656 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:22.526670 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:22.527531 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:22.527538 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:22.528843 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:22.528855 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:22.529397 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:22.529403 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:22.529602 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:22.530202 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:52:22.530219 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:52:22.531826 (+ 1607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:52:22.533736 (+ 1910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:22.533744 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:22.533761 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:22.534132 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:22.534395 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:22.534401 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:22.534406 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:22.534642 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:22.535025 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:52:22.535043 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:52:22.535285 (+ 242us) client_negotiation.cc:769] Sending connection context
0810 18:52:22.535472 (+ 187us) client_negotiation.cc:240] Negotiation successful
0810 18:52:22.535795 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":136,"threads_started":1}
I20260810 18:52:22.538003 7719 heartbeater.cc:347] Connected to a master server at 127.18.168.190:46049
I20260810 18:52:22.538244 7719 heartbeater.cc:464] Registering TS with master...
I20260810 18:52:22.538743 7719 heartbeater.cc:511] Master 127.18.168.190:46049 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.168.130@KRBTEST.COM; defaulting to no policy
I20260810 18:52:22.540351 7499 ts_manager.cc:194] Registered new tserver with Master: 88c77ad0619140a0afd9170c17ccb63b (127.18.168.129:43437)
I20260810 18:52:22.541604 7499 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.129@KRBTEST.COM'} at 127.18.168.129:55621
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab.
Entry for principal kudu/127.18.168.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab.
I20260810 18:52:22.558017 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.130:0
--local_ip_for_outbound_sockets=127.18.168.130
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:46049
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:44641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:52:22.670905 7724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:22.671186 7724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:22.671232 7724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:22.675580 7724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:22.675638 7724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:22.675735 7724 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.130
I20260810 18:52:22.681365 7724 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:44641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:46049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.130
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:22.682422 7724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:22.683493 7724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:22.693372 7729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:22.693322 7732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:22.693331 7730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:22.693714 7724 server_base.cc:1034] running on GCE node
I20260810 18:52:22.694087 7724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:22.694701 7724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:22.695878 7724 hybrid_clock.cc:648] HybridClock initialized: now 1786387942695839 us; error 46 us; skew 500 ppm
Aug 10 18:52:22 dist-test-slave-cbsf krb5kdc[7449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387942, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:22.698879 7724 init.cc:377] Logged in from keytab as kudu/127.18.168.130@KRBTEST.COM (short username kudu)
I20260810 18:52:22.699827 7724 webserver.cc:460] Webserver started at http://127.18.168.130:45955/ using document root <none> and password file <none>
I20260810 18:52:22.700403 7724 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:22.700496 7724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:22.700790 7724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:22.702801 7724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-1/data/instance:
uuid: "ca3c0217b0df46498b6046bf4385b35b"
format_stamp: "Formatted at 2026-08-10 18:52:22 on dist-test-slave-cbsf"
server_key: "510fae394c5011572f5f0568c109130c"
server_key_iv: "378409349db5d4fabb6cafd68b849d04"
server_key_version: "encryptionkey@0"
I20260810 18:52:22.703404 7724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance:
uuid: "ca3c0217b0df46498b6046bf4385b35b"
format_stamp: "Formatted at 2026-08-10 18:52:22 on dist-test-slave-cbsf"
server_key: "510fae394c5011572f5f0568c109130c"
server_key_iv: "378409349db5d4fabb6cafd68b849d04"
server_key_version: "encryptionkey@0"
I20260810 18:52:22.707343 7724 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260810 18:52:22.709846 7740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:22.711233 7724 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:52:22.711372 7724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-1/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-1/wal
uuid: "ca3c0217b0df46498b6046bf4385b35b"
format_stamp: "Formatted at 2026-08-10 18:52:22 on dist-test-slave-cbsf"
server_key: "510fae394c5011572f5f0568c109130c"
server_key_iv: "378409349db5d4fabb6cafd68b849d04"
server_key_version: "encryptionkey@0"
I20260810 18:52:22.711488 7724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:22.753705 7724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:22.757393 7724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:22.757606 7724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:22.758239 7724 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:52:22.759125 7724 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:52:22.759199 7724 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:22.759284 7724 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:52:22.759331 7724 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:22.768334 7724 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.130:34027
I20260810 18:52:22.768414 7868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.130:34027 every 8 connection(s)
I20260810 18:52:22.769328 7724 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
I20260810 18:52:22.775061 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 7724
I20260810 18:52:22.775202 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance
I20260810 18:52:22.775509 19106 external_mini_cluster.cc:1428] Setting key 7b258413667a3b7d05752f42eb233926
Aug 10 18:52:22 dist-test-slave-cbsf krb5kdc[7449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387942, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:52:22.785404 7545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:22.771216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:48027 (local address 127.18.168.190:46049)
0810 18:52:22.771388 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:22.771392 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:22.772332 (+ 940us) server_negotiation.cc:407] Connection header received
0810 18:52:22.773891 (+ 1559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:22.773894 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:22.773967 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:22.774081 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:22.775427 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:22.776106 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:22.777725 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:22.778017 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:22.778136 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:22.781190 (+ 3054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:22.781196 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:22.781203 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:22.781237 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:22.782984 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:22.783536 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:22.783541 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:22.783544 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:22.783647 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:22.784013 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:22.784020 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:22.784024 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:22.784321 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:22.784435 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:22.785053 (+ 618us) server_negotiation.cc:299] Negotiation successful
0810 18:52:22.785205 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 18:52:22.786154 7871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:22.771609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:46049 (local address 127.18.168.130:48027)
0810 18:52:22.772160 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:22.772216 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:22.773667 (+ 1451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:22.774266 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:22.774274 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:22.774651 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:52:22.775265 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:22.775296 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:22.776220 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:22.776224 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:22.777595 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:22.777606 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:22.778181 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:22.778185 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:22.778373 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:22.779088 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:52:22.779105 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:52:22.781013 (+ 1908us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:52:22.783116 (+ 2103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:22.783123 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:22.783145 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:22.783412 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:22.783744 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:22.783746 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:22.783748 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:22.783881 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:22.784439 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:52:22.784444 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:52:22.784755 (+ 311us) client_negotiation.cc:769] Sending connection context
0810 18:52:22.784930 (+ 175us) client_negotiation.cc:240] Negotiation successful
0810 18:52:22.785203 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":120,"threads_started":1}
I20260810 18:52:22.787429 7869 heartbeater.cc:347] Connected to a master server at 127.18.168.190:46049
WARNING: no policy specified for kudu/127.18.168.131@KRBTEST.COM; defaulting to no policy
I20260810 18:52:22.787703 7869 heartbeater.cc:464] Registering TS with master...
I20260810 18:52:22.788326 7869 heartbeater.cc:511] Master 127.18.168.190:46049 requested a full tablet report, sending...
I20260810 18:52:22.789489 7499 ts_manager.cc:194] Registered new tserver with Master: ca3c0217b0df46498b6046bf4385b35b (127.18.168.130:34027)
I20260810 18:52:22.790180 7499 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.130@KRBTEST.COM'} at 127.18.168.130:48027
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab.
Entry for principal kudu/127.18.168.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab.
I20260810 18:52:22.807865 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.131:0
--local_ip_for_outbound_sockets=127.18.168.131
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:46049
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:44641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:52:22.925489 7874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:22.925853 7874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:22.925940 7874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:22.930675 7874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:22.930764 7874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:22.930864 7874 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.131
I20260810 18:52:22.936430 7874 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:44641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:46049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.131
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:22.937618 7874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:22.938692 7874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:22.948859 7879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:22.948863 7882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:22.948838 7880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:22.948912 7874 server_base.cc:1034] running on GCE node
I20260810 18:52:22.949432 7874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:22.950088 7874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:22.951290 7874 hybrid_clock.cc:648] HybridClock initialized: now 1786387942951270 us; error 25 us; skew 500 ppm
Aug 10 18:52:22 dist-test-slave-cbsf krb5kdc[7449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387942, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:22.954102 7874 init.cc:377] Logged in from keytab as kudu/127.18.168.131@KRBTEST.COM (short username kudu)
I20260810 18:52:22.955000 7874 webserver.cc:460] Webserver started at http://127.18.168.131:33079/ using document root <none> and password file <none>
I20260810 18:52:22.955533 7874 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:22.955592 7874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:22.955761 7874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:22.957881 7874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-2/data/instance:
uuid: "8b45f51cb6734043b94faa699725d21e"
format_stamp: "Formatted at 2026-08-10 18:52:22 on dist-test-slave-cbsf"
server_key: "691ef76209d6a5f46e28c29419e3d3fa"
server_key_iv: "9bc51000862af3579ba9d8cf55c8c30d"
server_key_version: "encryptionkey@0"
I20260810 18:52:22.958411 7874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance:
uuid: "8b45f51cb6734043b94faa699725d21e"
format_stamp: "Formatted at 2026-08-10 18:52:22 on dist-test-slave-cbsf"
server_key: "691ef76209d6a5f46e28c29419e3d3fa"
server_key_iv: "9bc51000862af3579ba9d8cf55c8c30d"
server_key_version: "encryptionkey@0"
I20260810 18:52:22.962713 7874 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:52:22.965327 7890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:22.966693 7874 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:52:22.966833 7874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-2/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-2/wal
uuid: "8b45f51cb6734043b94faa699725d21e"
format_stamp: "Formatted at 2026-08-10 18:52:22 on dist-test-slave-cbsf"
server_key: "691ef76209d6a5f46e28c29419e3d3fa"
server_key_iv: "9bc51000862af3579ba9d8cf55c8c30d"
server_key_version: "encryptionkey@0"
I20260810 18:52:22.966950 7874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:22.994655 7874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:22.999303 7874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:22.999486 7874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:23.000063 7874 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:52:23.000993 7874 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:52:23.001042 7874 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:23.001083 7874 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:52:23.001104 7874 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:23.010758 7874 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.131:40493
I20260810 18:52:23.010780 8021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.131:40493 every 8 connection(s)
I20260810 18:52:23.011883 7874 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
I20260810 18:52:23.015631 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 7874
I20260810 18:52:23.015787 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance
I20260810 18:52:23.016101 19106 external_mini_cluster.cc:1428] Setting key 4334dd4823fc8fde4402e8be33c9f9d0
Aug 10 18:52:23 dist-test-slave-cbsf krb5kdc[7449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387942, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:52:23.026772 7545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:23.013848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:51219 (local address 127.18.168.190:46049)
0810 18:52:23.013998 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:23.014001 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:23.014925 (+ 924us) server_negotiation.cc:407] Connection header received
0810 18:52:23.016640 (+ 1715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:23.016644 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:23.016730 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:23.016829 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:23.018515 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:23.019060 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:23.020258 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:23.020569 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:23.020656 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:23.023241 (+ 2585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:23.023244 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:23.023247 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:23.023271 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:23.024900 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:23.025379 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:23.025382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:23.025383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:23.025498 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:23.025796 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:23.025798 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:23.025799 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:23.025978 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:23.026054 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:23.026549 (+ 495us) server_negotiation.cc:299] Negotiation successful
0810 18:52:23.026646 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 18:52:23.027621 8024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:23.014245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:46049 (local address 127.18.168.131:51219)
0810 18:52:23.014780 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:23.014826 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:23.016359 (+ 1533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:23.017083 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:23.017092 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:23.017596 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:52:23.018319 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:23.018340 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:23.019173 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:23.019176 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:23.020112 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:23.020122 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:23.020757 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:23.020761 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:23.020926 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:23.021490 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:52:23.021506 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:52:23.023083 (+ 1577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:52:23.025009 (+ 1926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:23.025013 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:23.025026 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:23.025289 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:23.025582 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:23.025584 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:23.025585 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:23.025717 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:23.026075 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:52:23.026080 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:52:23.026319 (+ 239us) client_negotiation.cc:769] Sending connection context
0810 18:52:23.026475 (+ 156us) client_negotiation.cc:240] Negotiation successful
0810 18:52:23.026728 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":117,"threads_started":1}
I20260810 18:52:23.029003 8022 heartbeater.cc:347] Connected to a master server at 127.18.168.190:46049
I20260810 18:52:23.029275 8022 heartbeater.cc:464] Registering TS with master...
I20260810 18:52:23.029826 8022 heartbeater.cc:511] Master 127.18.168.190:46049 requested a full tablet report, sending...
I20260810 18:52:23.030894 7499 ts_manager.cc:194] Registered new tserver with Master: 8b45f51cb6734043b94faa699725d21e (127.18.168.131:40493)
I20260810 18:52:23.031476 7499 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.131@KRBTEST.COM'} at 127.18.168.131:51219
I20260810 18:52:23.042357 19106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:52:23.052484 7545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:23.044570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39978 (local address 127.18.168.190:46049)
0810 18:52:23.044770 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:23.044773 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:23.044906 (+ 133us) server_negotiation.cc:407] Connection header received
0810 18:52:23.045011 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:23.045013 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:23.045054 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:23.045134 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:23.045835 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:23.046397 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:23.047245 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:23.047490 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:23.047580 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:23.049138 (+ 1558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:23.049141 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:23.049143 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:23.049169 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:23.050777 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:23.051315 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:23.051318 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:23.051320 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:23.051398 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:23.051694 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:23.051696 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:23.051697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:23.051911 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:23.052031 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:23.052271 (+ 240us) server_negotiation.cc:299] Negotiation successful
0810 18:52:23.052358 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 18:52:23.055686 7499 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39978:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:52:23.058911 7499 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:52:23.074397 8038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:23.068457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.131:40493 (local address 127.0.0.1:35130)
0810 18:52:23.069450 (+ 993us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:23.069466 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:23.069576 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:23.070095 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:23.070098 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:23.070118 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:23.070200 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:23.070207 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:23.071568 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:23.071573 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:23.072854 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:23.072867 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:23.073738 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:23.073743 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:23.073872 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:23.073940 (+ 68us) client_negotiation.cc:769] Sending connection context
0810 18:52:23.074071 (+ 131us) client_negotiation.cc:240] Negotiation successful
0810 18:52:23.074209 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":914,"thread_start_us":117,"threads_started":1}
I20260810 18:52:23.074632 8039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:23.068996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35130 (local address 127.18.168.131:40493)
0810 18:52:23.069355 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:23.069367 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:23.069551 (+ 184us) server_negotiation.cc:407] Connection header received
0810 18:52:23.069734 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:23.069742 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:23.069943 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:23.070111 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:23.070304 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:23.071399 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:23.072973 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:23.073594 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:23.073754 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:23.073896 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:23.074107 (+ 211us) server_negotiation.cc:299] Negotiation successful
0810 18:52:23.074228 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":125,"threads_started":1}
I20260810 18:52:23.075098 8035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:23.067874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58630 (local address 127.18.168.129:43437)
0810 18:52:23.068602 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:23.068610 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:23.068636 (+ 26us) server_negotiation.cc:407] Connection header received
0810 18:52:23.068940 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:23.068947 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:23.069175 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:23.069349 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:23.069780 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:23.071005 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:23.072682 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:23.073593 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:23.073727 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:23.073888 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:23.074395 (+ 507us) server_negotiation.cc:299] Negotiation successful
0810 18:52:23.074929 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":139,"threads_started":1}
I20260810 18:52:23.075647 8037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:23.068149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33006 (local address 127.18.168.130:34027)
0810 18:52:23.068907 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:23.068921 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:23.069289 (+ 368us) server_negotiation.cc:407] Connection header received
0810 18:52:23.069375 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:23.069383 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:23.069560 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:23.069705 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:23.070002 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:23.071009 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:23.072924 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:23.073744 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:23.073853 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:23.073963 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:23.074924 (+ 961us) server_negotiation.cc:299] Negotiation successful
0810 18:52:23.075052 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":125,"threads_started":1}
I20260810 18:52:23.075820 8036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:23.067963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:34027 (local address 127.0.0.1:33006)
0810 18:52:23.069068 (+ 1105us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:23.069082 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:23.069184 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:23.069787 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:23.069790 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:23.069806 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:23.069894 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:23.069899 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:23.071193 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:23.071197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:23.072768 (+ 1571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:23.072781 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:23.073984 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:23.073988 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:23.074149 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:23.074175 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 18:52:23.074251 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 18:52:23.074318 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1032,"mutex_wait_us":66,"thread_start_us":124,"threads_started":1}
I20260810 18:52:23.075858 8034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:23.067670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:43437 (local address 127.0.0.1:58630)
0810 18:52:23.068427 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:23.068482 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:23.068626 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:23.069447 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:23.069453 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:23.069478 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:23.069649 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:23.069661 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:23.071298 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:23.071302 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:23.072554 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:23.072564 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:23.073956 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:23.073960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:23.074087 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:23.074135 (+ 48us) client_negotiation.cc:769] Sending connection context
0810 18:52:23.074206 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 18:52:23.074271 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"mutex_wait_us":165,"thread_start_us":112,"threads_started":1}
I20260810 18:52:23.077837 7948 tablet_service.cc:1467] Processing CreateTablet for tablet e44821e3642140809300389c9c929f93 (DEFAULT_TABLE table=test-table [id=6ce84c8e573448bf85c41889b30d4730]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:52:23.078598 7644 tablet_service.cc:1467] Processing CreateTablet for tablet e44821e3642140809300389c9c929f93 (DEFAULT_TABLE table=test-table [id=6ce84c8e573448bf85c41889b30d4730]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:52:23.078598 7794 tablet_service.cc:1467] Processing CreateTablet for tablet e44821e3642140809300389c9c929f93 (DEFAULT_TABLE table=test-table [id=6ce84c8e573448bf85c41889b30d4730]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:52:23.079182 7948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e44821e3642140809300389c9c929f93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:23.079617 7794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e44821e3642140809300389c9c929f93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:23.079617 7644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e44821e3642140809300389c9c929f93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:23.086203 8040 tablet_bootstrap.cc:492] T e44821e3642140809300389c9c929f93 P 8b45f51cb6734043b94faa699725d21e: Bootstrap starting.
I20260810 18:52:23.087512 8042 tablet_bootstrap.cc:492] T e44821e3642140809300389c9c929f93 P 88c77ad0619140a0afd9170c17ccb63b: Bootstrap starting.
I20260810 18:52:23.088907 8040 tablet_bootstrap.cc:654] T e44821e3642140809300389c9c929f93 P 8b45f51cb6734043b94faa699725d21e: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:23.089066 8043 tablet_bootstrap.cc:492] T e44821e3642140809300389c9c929f93 P ca3c0217b0df46498b6046bf4385b35b: Bootstrap starting.
I20260810 18:52:23.089432 8042 tablet_bootstrap.cc:654] T e44821e3642140809300389c9c929f93 P 88c77ad0619140a0afd9170c17ccb63b: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:23.090356 8042 log.cc:824] T e44821e3642140809300389c9c929f93 P 88c77ad0619140a0afd9170c17ccb63b: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:23.091137 8043 tablet_bootstrap.cc:654] T e44821e3642140809300389c9c929f93 P ca3c0217b0df46498b6046bf4385b35b: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:23.091120 8040 log.cc:824] T e44821e3642140809300389c9c929f93 P 8b45f51cb6734043b94faa699725d21e: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:23.092449 8043 log.cc:824] T e44821e3642140809300389c9c929f93 P ca3c0217b0df46498b6046bf4385b35b: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:23.092871 8042 tablet_bootstrap.cc:492] T e44821e3642140809300389c9c929f93 P 88c77ad0619140a0afd9170c17ccb63b: No bootstrap required, opened a new log
I20260810 18:52:23.093079 8042 ts_tablet_manager.cc:1397] T e44821e3642140809300389c9c929f93 P 88c77ad0619140a0afd9170c17ccb63b: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260810 18:52:23.093132 8040 tablet_bootstrap.cc:492] T e44821e3642140809300389c9c929f93 P 8b45f51cb6734043b94faa699725d21e: No bootstrap required, opened a new log
I20260810 18:52:23.093333 8040 ts_tablet_manager.cc:1397] T e44821e3642140809300389c9c929f93 P 8b45f51cb6734043b94faa699725d21e: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260810 18:52:23.094477 8043 tablet_bootstrap.cc:492] T e44821e3642140809300389c9c929f93 P ca3c0217b0df46498b6046bf4385b35b: No bootstrap required, opened a new log
I20260810 18:52:23.094658 8043 ts_tablet_manager.cc:1397] T e44821e3642140809300389c9c929f93 P ca3c0217b0df46498b6046bf4385b35b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 18:52:23.096602 8040 raft_consensus.cc:348] T e44821e3642140809300389c9c929f93 P 8b45f51cb6734043b94faa699725d21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88c77ad0619140a0afd9170c17ccb63b" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43437 } } peers { permanent_uuid: "8b45f51cb6734043b94faa699725d21e" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 40493 } } peers { permanent_uuid: "ca3c0217b0df46498b6046bf4385b35b" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 34027 } }
I20260810 18:52:23.096930 8040 raft_consensus.cc:374] T e44821e3642140809300389c9c929f93 P 8b45f51cb6734043b94faa699725d21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:23.096997 8040 raft_consensus.cc:729] T e44821e3642140809300389c9c929f93 P 8b45f51cb6734043b94faa699725d21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b45f51cb6734043b94faa699725d21e, State: Initialized, Role: FOLLOWER
I20260810 18:52:23.097697 8042 raft_consensus.cc:348] T e44821e3642140809300389c9c929f93 P 88c77ad0619140a0afd9170c17ccb63b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88c77ad0619140a0afd9170c17ccb63b" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43437 } } peers { permanent_uuid: "8b45f51cb6734043b94faa699725d21e" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 40493 } } peers { permanent_uuid: "ca3c0217b0df46498b6046bf4385b35b" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 34027 } }
I20260810 18:52:23.097962 8042 raft_consensus.cc:374] T e44821e3642140809300389c9c929f93 P 88c77ad0619140a0afd9170c17ccb63b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:23.098011 8042 raft_consensus.cc:729] T e44821e3642140809300389c9c929f93 P 88c77ad0619140a0afd9170c17ccb63b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88c77ad0619140a0afd9170c17ccb63b, State: Initialized, Role: FOLLOWER
I20260810 18:52:23.097889 8040 consensus_queue.cc:260] T e44821e3642140809300389c9c929f93 P 8b45f51cb6734043b94faa699725d21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88c77ad0619140a0afd9170c17ccb63b" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43437 } } peers { permanent_uuid: "8b45f51cb6734043b94faa699725d21e" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 40493 } } peers { permanent_uuid: "ca3c0217b0df46498b6046bf4385b35b" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 34027 } }
I20260810 18:52:23.098496 8042 consensus_queue.cc:260] T e44821e3642140809300389c9c929f93 P 88c77ad0619140a0afd9170c17ccb63b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88c77ad0619140a0afd9170c17ccb63b" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43437 } } peers { permanent_uuid: "8b45f51cb6734043b94faa699725d21e" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 40493 } } peers { permanent_uuid: "ca3c0217b0df46498b6046bf4385b35b" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 34027 } }
I20260810 18:52:23.098711 8022 heartbeater.cc:503] Master 127.18.168.190:46049 was elected leader, sending a full tablet report...
I20260810 18:52:23.098784 8043 raft_consensus.cc:348] T e44821e3642140809300389c9c929f93 P ca3c0217b0df46498b6046bf4385b35b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88c77ad0619140a0afd9170c17ccb63b" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43437 } } peers { permanent_uuid: "8b45f51cb6734043b94faa699725d21e" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 40493 } } peers { permanent_uuid: "ca3c0217b0df46498b6046bf4385b35b" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 34027 } }
I20260810 18:52:23.099020 8043 raft_consensus.cc:374] T e44821e3642140809300389c9c929f93 P ca3c0217b0df46498b6046bf4385b35b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:23.099027 8040 ts_tablet_manager.cc:1428] T e44821e3642140809300389c9c929f93 P 8b45f51cb6734043b94faa699725d21e: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260810 18:52:23.099092 8043 raft_consensus.cc:729] T e44821e3642140809300389c9c929f93 P ca3c0217b0df46498b6046bf4385b35b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca3c0217b0df46498b6046bf4385b35b, State: Initialized, Role: FOLLOWER
I20260810 18:52:23.099527 8043 consensus_queue.cc:260] T e44821e3642140809300389c9c929f93 P ca3c0217b0df46498b6046bf4385b35b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88c77ad0619140a0afd9170c17ccb63b" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43437 } } peers { permanent_uuid: "8b45f51cb6734043b94faa699725d21e" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 40493 } } peers { permanent_uuid: "ca3c0217b0df46498b6046bf4385b35b" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 34027 } }
I20260810 18:52:23.099748 8042 ts_tablet_manager.cc:1428] T e44821e3642140809300389c9c929f93 P 88c77ad0619140a0afd9170c17ccb63b: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260810 18:52:23.100315 7869 heartbeater.cc:503] Master 127.18.168.190:46049 was elected leader, sending a full tablet report...
I20260810 18:52:23.100492 8043 ts_tablet_manager.cc:1428] T e44821e3642140809300389c9c929f93 P ca3c0217b0df46498b6046bf4385b35b: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 18:52:23.099937 7719 heartbeater.cc:503] Master 127.18.168.190:46049 was elected leader, sending a full tablet report...
W20260810 18:52:23.262907 8023 tablet.cc:2369] T e44821e3642140809300389c9c929f93 P 8b45f51cb6734043b94faa699725d21e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:52:23.270669 7870 tablet.cc:2369] T e44821e3642140809300389c9c929f93 P ca3c0217b0df46498b6046bf4385b35b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:52:23.272547 7720 tablet.cc:2369] T e44821e3642140809300389c9c929f93 P 88c77ad0619140a0afd9170c17ccb63b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:52:23.340436 8049 raft_consensus.cc:482] T e44821e3642140809300389c9c929f93 P 8b45f51cb6734043b94faa699725d21e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:52:23.340715 8049 raft_consensus.cc:504] T e44821e3642140809300389c9c929f93 P 8b45f51cb6734043b94faa699725d21e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88c77ad0619140a0afd9170c17ccb63b" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43437 } } peers { permanent_uuid: "8b45f51cb6734043b94faa699725d21e" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 40493 } } peers { permanent_uuid: "ca3c0217b0df46498b6046bf4385b35b" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 34027 } }
I20260810 18:52:23.342046 8049 leader_election.cc:288] T e44821e3642140809300389c9c929f93 P 8b45f51cb6734043b94faa699725d21e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88c77ad0619140a0afd9170c17ccb63b (127.18.168.129:43437), ca3c0217b0df46498b6046bf4385b35b (127.18.168.130:34027)
I20260810 18:52:23.346488 8024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:23.342331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:34027 (local address 127.18.168.131:52993)
0810 18:52:23.342542 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:23.342562 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:23.342702 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:23.342996 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:23.342999 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:23.343019 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:23.343148 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:23.343154 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:23.344052 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:23.344058 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:23.345058 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:23.345068 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:23.346032 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:23.346035 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:23.346145 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:23.346176 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 18:52:23.346255 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 18:52:23.346329 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"mutex_wait_us":1}
I20260810 18:52:23.346558 8037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:23.342502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:52993 (local address 127.18.168.130:34027)
0810 18:52:23.342675 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:23.342680 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:23.342698 (+ 18us) server_negotiation.cc:407] Connection header received
0810 18:52:23.342814 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:23.342817 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:23.342872 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:23.342970 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:23.343268 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:23.343928 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:23.345198 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:23.345909 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:23.346052 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:23.346136 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:23.346299 (+ 163us) server_negotiation.cc:299] Negotiation successful
0810 18:52:23.346415 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 18:52:23.346608 8035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:23.342506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:44771 (local address 127.18.168.129:43437)
0810 18:52:23.342661 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:23.342666 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:23.343042 (+ 376us) server_negotiation.cc:407] Connection header received
0810 18:52:23.343210 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:23.343213 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:23.343262 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:23.343346 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:23.343671 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:23.344238 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:23.345342 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:23.346069 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:23.346202 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:23.346276 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:23.346412 (+ 136us) server_negotiation.cc:299] Negotiation successful
0810 18:52:23.346491 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 18:52:23.347056 8052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:23.342335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:43437 (local address 127.18.168.131:44771)
0810 18:52:23.342943 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:23.342955 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:23.343099 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:23.343356 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:23.343358 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:23.343374 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:23.343530 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:23.343537 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:23.344353 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:23.344357 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:23.345185 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:23.345195 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:23.346186 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:23.346189 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:23.346292 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:23.346311 (+ 19us) client_negotiation.cc:769] Sending connection context
0810 18:52:23.346378 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 18:52:23.346433 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"mutex_wait_us":229,"thread_start_us":176,"threads_started":1}
I20260810 18:52:23.347312 7817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e44821e3642140809300389c9c929f93" candidate_uuid: "8b45f51cb6734043b94faa699725d21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca3c0217b0df46498b6046bf4385b35b" is_pre_election: true
I20260810 18:52:23.347731 7817 raft_consensus.cc:2454] T e44821e3642140809300389c9c929f93 P ca3c0217b0df46498b6046bf4385b35b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b45f51cb6734043b94faa699725d21e in term 0.
I20260810 18:52:23.347822 7665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e44821e3642140809300389c9c929f93" candidate_uuid: "8b45f51cb6734043b94faa699725d21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88c77ad0619140a0afd9170c17ccb63b" is_pre_election: true
I20260810 18:52:23.348132 7665 raft_consensus.cc:2454] T e44821e3642140809300389c9c929f93 P 88c77ad0619140a0afd9170c17ccb63b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b45f51cb6734043b94faa699725d21e in term 0.
I20260810 18:52:23.348333 7892 leader_election.cc:302] T e44821e3642140809300389c9c929f93 P 8b45f51cb6734043b94faa699725d21e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b45f51cb6734043b94faa699725d21e, ca3c0217b0df46498b6046bf4385b35b; no voters:
I20260810 18:52:23.348658 8049 raft_consensus.cc:2781] T e44821e3642140809300389c9c929f93 P 8b45f51cb6734043b94faa699725d21e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:52:23.348764 8049 raft_consensus.cc:482] T e44821e3642140809300389c9c929f93 P 8b45f51cb6734043b94faa699725d21e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:52:23.348837 8049 raft_consensus.cc:3037] T e44821e3642140809300389c9c929f93 P 8b45f51cb6734043b94faa699725d21e [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:23.349990 8049 raft_consensus.cc:504] T e44821e3642140809300389c9c929f93 P 8b45f51cb6734043b94faa699725d21e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88c77ad0619140a0afd9170c17ccb63b" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43437 } } peers { permanent_uuid: "8b45f51cb6734043b94faa699725d21e" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 40493 } } peers { permanent_uuid: "ca3c0217b0df46498b6046bf4385b35b" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 34027 } }
I20260810 18:52:23.350356 8049 leader_election.cc:288] T e44821e3642140809300389c9c929f93 P 8b45f51cb6734043b94faa699725d21e [CANDIDATE]: Term 1 election: Requested vote from peers 88c77ad0619140a0afd9170c17ccb63b (127.18.168.129:43437), ca3c0217b0df46498b6046bf4385b35b (127.18.168.130:34027)
I20260810 18:52:23.350783 7665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e44821e3642140809300389c9c929f93" candidate_uuid: "8b45f51cb6734043b94faa699725d21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88c77ad0619140a0afd9170c17ccb63b"
I20260810 18:52:23.350783 7817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e44821e3642140809300389c9c929f93" candidate_uuid: "8b45f51cb6734043b94faa699725d21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca3c0217b0df46498b6046bf4385b35b"
I20260810 18:52:23.350935 7665 raft_consensus.cc:3037] T e44821e3642140809300389c9c929f93 P 88c77ad0619140a0afd9170c17ccb63b [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:23.350960 7817 raft_consensus.cc:3037] T e44821e3642140809300389c9c929f93 P ca3c0217b0df46498b6046bf4385b35b [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:23.352376 7817 raft_consensus.cc:2454] T e44821e3642140809300389c9c929f93 P ca3c0217b0df46498b6046bf4385b35b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b45f51cb6734043b94faa699725d21e in term 1.
I20260810 18:52:23.352376 7665 raft_consensus.cc:2454] T e44821e3642140809300389c9c929f93 P 88c77ad0619140a0afd9170c17ccb63b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b45f51cb6734043b94faa699725d21e in term 1.
I20260810 18:52:23.352788 7894 leader_election.cc:302] T e44821e3642140809300389c9c929f93 P 8b45f51cb6734043b94faa699725d21e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 88c77ad0619140a0afd9170c17ccb63b, 8b45f51cb6734043b94faa699725d21e, ca3c0217b0df46498b6046bf4385b35b; no voters:
I20260810 18:52:23.353040 8049 raft_consensus.cc:2781] T e44821e3642140809300389c9c929f93 P 8b45f51cb6734043b94faa699725d21e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:52:23.353308 8049 raft_consensus.cc:686] T e44821e3642140809300389c9c929f93 P 8b45f51cb6734043b94faa699725d21e [term 1 LEADER]: Becoming Leader. State: Replica: 8b45f51cb6734043b94faa699725d21e, State: Running, Role: LEADER
I20260810 18:52:23.353700 8049 consensus_queue.cc:237] T e44821e3642140809300389c9c929f93 P 8b45f51cb6734043b94faa699725d21e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88c77ad0619140a0afd9170c17ccb63b" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43437 } } peers { permanent_uuid: "8b45f51cb6734043b94faa699725d21e" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 40493 } } peers { permanent_uuid: "ca3c0217b0df46498b6046bf4385b35b" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 34027 } }
I20260810 18:52:23.357190 7499 catalog_manager.cc:5495] T e44821e3642140809300389c9c929f93 P 8b45f51cb6734043b94faa699725d21e reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b45f51cb6734043b94faa699725d21e (127.18.168.131). New cstate: current_term: 1 leader_uuid: "8b45f51cb6734043b94faa699725d21e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88c77ad0619140a0afd9170c17ccb63b" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b45f51cb6734043b94faa699725d21e" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 40493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca3c0217b0df46498b6046bf4385b35b" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 34027 } health_report { overall_health: UNKNOWN } } }
I20260810 18:52:23.368005 8039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:23.364806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35144 (local address 127.18.168.131:40493)
0810 18:52:23.364932 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:23.364935 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:23.364949 (+ 14us) server_negotiation.cc:407] Connection header received
0810 18:52:23.365064 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:23.365066 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:23.365117 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:23.365206 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:52:23.365421 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:23.365940 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:23.366738 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:23.366944 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:23.367018 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:23.367291 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:52:23.367679 (+ 388us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:52:23.367785 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:23.367863 (+ 78us) server_negotiation.cc:299] Negotiation successful
0810 18:52:23.367897 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 18:52:23.373982 7817 raft_consensus.cc:1261] T e44821e3642140809300389c9c929f93 P ca3c0217b0df46498b6046bf4385b35b [term 1 FOLLOWER]: Refusing update from remote peer 8b45f51cb6734043b94faa699725d21e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:52:23.373983 7665 raft_consensus.cc:1261] T e44821e3642140809300389c9c929f93 P 88c77ad0619140a0afd9170c17ccb63b [term 1 FOLLOWER]: Refusing update from remote peer 8b45f51cb6734043b94faa699725d21e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:52:23.374737 8054 consensus_queue.cc:1035] T e44821e3642140809300389c9c929f93 P 8b45f51cb6734043b94faa699725d21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca3c0217b0df46498b6046bf4385b35b" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 34027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:52:23.374928 8049 consensus_queue.cc:1035] T e44821e3642140809300389c9c929f93 P 8b45f51cb6734043b94faa699725d21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88c77ad0619140a0afd9170c17ccb63b" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 43437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:52:23.381938 8059 mvcc.cc:205] Tried to move back new op lower bound from 7317045016052559872 to 7317045015979335680. Current Snapshot: MvccSnapshot[applied={T|T < 7317045016052559872}]
I20260810 18:52:23.385578 8062 mvcc.cc:205] Tried to move back new op lower bound from 7317045016052559872 to 7317045015979335680. Current Snapshot: MvccSnapshot[applied={T|T < 7317045016052559872}]
I20260810 18:52:23.391260 7498 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39978:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:52:23.391454 7498 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39978:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:52:23.394078 7498 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8555d45ad5c04fecad35555aeb003541: Sending DeleteTablet for 3 replicas of tablet e44821e3642140809300389c9c929f93
I20260810 18:52:23.394968 7948 tablet_service.cc:1514] Processing DeleteTablet for tablet e44821e3642140809300389c9c929f93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:52:23 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:35130
I20260810 18:52:23.395007 7644 tablet_service.cc:1514] Processing DeleteTablet for tablet e44821e3642140809300389c9c929f93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:52:23 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:58630
I20260810 18:52:23.394994 7794 tablet_service.cc:1514] Processing DeleteTablet for tablet e44821e3642140809300389c9c929f93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:52:23 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:33006
I20260810 18:52:23.395505 8068 tablet_replica.cc:330] T e44821e3642140809300389c9c929f93 P ca3c0217b0df46498b6046bf4385b35b: stopping tablet replica
I20260810 18:52:23.395656 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 7569
I20260810 18:52:23.395828 8069 tablet_replica.cc:330] T e44821e3642140809300389c9c929f93 P 88c77ad0619140a0afd9170c17ccb63b: stopping tablet replica
I20260810 18:52:23.395896 8068 raft_consensus.cc:2229] T e44821e3642140809300389c9c929f93 P ca3c0217b0df46498b6046bf4385b35b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:52:23.396065 8070 tablet_replica.cc:330] T e44821e3642140809300389c9c929f93 P 8b45f51cb6734043b94faa699725d21e: stopping tablet replica
I20260810 18:52:23.396227 8068 raft_consensus.cc:2258] T e44821e3642140809300389c9c929f93 P ca3c0217b0df46498b6046bf4385b35b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:52:23.396421 8070 raft_consensus.cc:2229] T e44821e3642140809300389c9c929f93 P 8b45f51cb6734043b94faa699725d21e [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:52:23.397009 8070 raft_consensus.cc:2258] T e44821e3642140809300389c9c929f93 P 8b45f51cb6734043b94faa699725d21e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:52:23.398753 8070 ts_tablet_manager.cc:1905] T e44821e3642140809300389c9c929f93 P 8b45f51cb6734043b94faa699725d21e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:52:23.398844 8068 ts_tablet_manager.cc:1905] T e44821e3642140809300389c9c929f93 P ca3c0217b0df46498b6046bf4385b35b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:52:23.401738 8070 ts_tablet_manager.cc:1918] T e44821e3642140809300389c9c929f93 P 8b45f51cb6734043b94faa699725d21e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:52:23.401924 8070 log.cc:1196] T e44821e3642140809300389c9c929f93 P 8b45f51cb6734043b94faa699725d21e: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-2/wal/wals/e44821e3642140809300389c9c929f93
I20260810 18:52:23.402386 8070 ts_tablet_manager.cc:1939] T e44821e3642140809300389c9c929f93 P 8b45f51cb6734043b94faa699725d21e: Deleting consensus metadata
I20260810 18:52:23.402640 8068 ts_tablet_manager.cc:1918] T e44821e3642140809300389c9c929f93 P ca3c0217b0df46498b6046bf4385b35b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:52:23.402781 8068 log.cc:1196] T e44821e3642140809300389c9c929f93 P ca3c0217b0df46498b6046bf4385b35b: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387687857264-19106-0/minicluster-data/ts-1/wal/wals/e44821e3642140809300389c9c929f93
I20260810 18:52:23.403126 8068 ts_tablet_manager.cc:1939] T e44821e3642140809300389c9c929f93 P ca3c0217b0df46498b6046bf4385b35b: Deleting consensus metadata
I20260810 18:52:23.403999 7483 catalog_manager.cc:4841] TS 8b45f51cb6734043b94faa699725d21e (127.18.168.131:40493): tablet e44821e3642140809300389c9c929f93 (table test-table [id=6ce84c8e573448bf85c41889b30d4730]) successfully deleted
I20260810 18:52:23.404963 7480 catalog_manager.cc:4841] TS ca3c0217b0df46498b6046bf4385b35b (127.18.168.130:34027): tablet e44821e3642140809300389c9c929f93 (table test-table [id=6ce84c8e573448bf85c41889b30d4730]) successfully deleted
W20260810 18:52:23.412369 7483 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.168.129:43437 (error 108)
I20260810 18:52:23.413194 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 7724
I20260810 18:52:23.415601 8036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:23.413270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:43437 (local address 127.0.0.1:58632)
0810 18:52:23.415405 (+ 2135us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:23.415496 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.129:43437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2023}
W20260810 18:52:23.417011 7483 catalog_manager.cc:4584] TS 88c77ad0619140a0afd9170c17ccb63b (127.18.168.129:43437): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e44821e3642140809300389c9c929f93: Network error: Client connection negotiation failed: client connection to 127.18.168.129:43437: connect: Connection refused (error 111)
I20260810 18:52:23.426086 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 7874
I20260810 18:52:23.434749 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 7463
2026-08-10T18:52:23Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2612 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-10T18:52:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:52:23Z Disabled control of system clock
I20260810 18:52:23.469861 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:43707
--webserver_interface=127.18.168.190
--webserver_port=0
--builtin_ntp_servers=127.18.168.148:38081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:43707
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 18:52:23.583968 8075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:23.584239 8075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:23.584306 8075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:23.588438 8075 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:52:23.588497 8075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:23.588513 8075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:23.588559 8075 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:52:23.588593 8075 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:52:23.593349 8075 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:38081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:43707
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:43707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:23.594458 8075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:23.595943 8075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:23.604532 8083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:23.604614 8086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:23.604502 8084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:23.604895 8075 server_base.cc:1034] running on GCE node
I20260810 18:52:23.605562 8075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:23.606671 8075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:23.607872 8075 hybrid_clock.cc:648] HybridClock initialized: now 1786387943607841 us; error 39 us; skew 500 ppm
I20260810 18:52:23.609836 8075 webserver.cc:460] Webserver started at http://127.18.168.190:43271/ using document root <none> and password file <none>
I20260810 18:52:23.610461 8075 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:23.610558 8075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:23.610805 8075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:23.613061 8075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "555cf7d9b9714c26baa8aebe456d4e29"
format_stamp: "Formatted at 2026-08-10 18:52:23 on dist-test-slave-cbsf"
server_key: "3c3a59eb211b7047b8f5aafcaa9c2fd0"
server_key_iv: "4da836e338a0647b96f2b6d2457edd7d"
server_key_version: "encryptionkey@0"
I20260810 18:52:23.613641 8075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "555cf7d9b9714c26baa8aebe456d4e29"
format_stamp: "Formatted at 2026-08-10 18:52:23 on dist-test-slave-cbsf"
server_key: "3c3a59eb211b7047b8f5aafcaa9c2fd0"
server_key_iv: "4da836e338a0647b96f2b6d2457edd7d"
server_key_version: "encryptionkey@0"
I20260810 18:52:23.617414 8075 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:52:23.619880 8092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:23.621207 8075 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:52:23.621349 8075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "555cf7d9b9714c26baa8aebe456d4e29"
format_stamp: "Formatted at 2026-08-10 18:52:23 on dist-test-slave-cbsf"
server_key: "3c3a59eb211b7047b8f5aafcaa9c2fd0"
server_key_iv: "4da836e338a0647b96f2b6d2457edd7d"
server_key_version: "encryptionkey@0"
I20260810 18:52:23.621462 8075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:23.642257 8075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:23.642979 8075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:23.643191 8075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:23.650601 8075 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:43707
I20260810 18:52:23.650614 8154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:43707 every 8 connection(s)
I20260810 18:52:23.651538 8075 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:52:23.654726 8155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:23.656744 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 8075
I20260810 18:52:23.656881 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:52:23.657239 19106 external_mini_cluster.cc:1428] Setting key 161073c10b315a6d92df80d680b605fa
I20260810 18:52:23.661185 8155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 555cf7d9b9714c26baa8aebe456d4e29: Bootstrap starting.
I20260810 18:52:23.663388 8155 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 555cf7d9b9714c26baa8aebe456d4e29: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:23.664166 8155 log.cc:824] T 00000000000000000000000000000000 P 555cf7d9b9714c26baa8aebe456d4e29: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:23.666070 8155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 555cf7d9b9714c26baa8aebe456d4e29: No bootstrap required, opened a new log
I20260810 18:52:23.667582 8158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:23.658799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51996 (local address 127.18.168.190:43707)
0810 18:52:23.659595 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:23.659613 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:23.659651 (+ 38us) server_negotiation.cc:407] Connection header received
0810 18:52:23.660334 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:23.660350 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:23.660916 (+ 566us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:23.661268 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:23.661743 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:23.662830 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:23.664119 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:23.664747 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:23.664921 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:23.665833 (+ 912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:23.665846 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:23.665860 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:52:23.666305 (+ 445us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:23.666363 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:52:23.666384 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:23.666506 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:23.666631 (+ 125us) server_negotiation.cc:299] Negotiation successful
0810 18:52:23.666891 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":139,"threads_started":1}
I20260810 18:52:23.669257 8155 raft_consensus.cc:348] T 00000000000000000000000000000000 P 555cf7d9b9714c26baa8aebe456d4e29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "555cf7d9b9714c26baa8aebe456d4e29" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43707 } }
I20260810 18:52:23.669507 8155 raft_consensus.cc:374] T 00000000000000000000000000000000 P 555cf7d9b9714c26baa8aebe456d4e29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:23.669587 8155 raft_consensus.cc:729] T 00000000000000000000000000000000 P 555cf7d9b9714c26baa8aebe456d4e29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 555cf7d9b9714c26baa8aebe456d4e29, State: Initialized, Role: FOLLOWER
I20260810 18:52:23.670014 8155 consensus_queue.cc:260] T 00000000000000000000000000000000 P 555cf7d9b9714c26baa8aebe456d4e29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "555cf7d9b9714c26baa8aebe456d4e29" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43707 } }
I20260810 18:52:23.670177 8155 raft_consensus.cc:388] T 00000000000000000000000000000000 P 555cf7d9b9714c26baa8aebe456d4e29 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:52:23.670248 8155 raft_consensus.cc:482] T 00000000000000000000000000000000 P 555cf7d9b9714c26baa8aebe456d4e29 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:52:23.670349 8155 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 555cf7d9b9714c26baa8aebe456d4e29 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:23.671240 8155 raft_consensus.cc:504] T 00000000000000000000000000000000 P 555cf7d9b9714c26baa8aebe456d4e29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "555cf7d9b9714c26baa8aebe456d4e29" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43707 } }
I20260810 18:52:23.671536 8155 leader_election.cc:302] T 00000000000000000000000000000000 P 555cf7d9b9714c26baa8aebe456d4e29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 555cf7d9b9714c26baa8aebe456d4e29; no voters:
I20260810 18:52:23.671823 8155 leader_election.cc:288] T 00000000000000000000000000000000 P 555cf7d9b9714c26baa8aebe456d4e29 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:52:23.671952 8160 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 555cf7d9b9714c26baa8aebe456d4e29 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:52:23.672228 8160 raft_consensus.cc:686] T 00000000000000000000000000000000 P 555cf7d9b9714c26baa8aebe456d4e29 [term 1 LEADER]: Becoming Leader. State: Replica: 555cf7d9b9714c26baa8aebe456d4e29, State: Running, Role: LEADER
I20260810 18:52:23.672549 8160 consensus_queue.cc:237] T 00000000000000000000000000000000 P 555cf7d9b9714c26baa8aebe456d4e29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "555cf7d9b9714c26baa8aebe456d4e29" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43707 } }
I20260810 18:52:23.673066 8155 sys_catalog.cc:564] T 00000000000000000000000000000000 P 555cf7d9b9714c26baa8aebe456d4e29 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:52:23.674342 8162 sys_catalog.cc:455] T 00000000000000000000000000000000 P 555cf7d9b9714c26baa8aebe456d4e29 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 555cf7d9b9714c26baa8aebe456d4e29. Latest consensus state: current_term: 1 leader_uuid: "555cf7d9b9714c26baa8aebe456d4e29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "555cf7d9b9714c26baa8aebe456d4e29" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43707 } } }
I20260810 18:52:23.674479 8162 sys_catalog.cc:458] T 00000000000000000000000000000000 P 555cf7d9b9714c26baa8aebe456d4e29 [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:23.674329 8161 sys_catalog.cc:455] T 00000000000000000000000000000000 P 555cf7d9b9714c26baa8aebe456d4e29 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "555cf7d9b9714c26baa8aebe456d4e29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "555cf7d9b9714c26baa8aebe456d4e29" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43707 } } }
I20260810 18:52:23.674765 8161 sys_catalog.cc:458] T 00000000000000000000000000000000 P 555cf7d9b9714c26baa8aebe456d4e29 [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:23.675025 8169 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:52:23.678062 8169 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:52:23.683871 8169 catalog_manager.cc:1347] Generated new cluster ID: 9f02463532c64c5e9219cf1a97086bb6
I20260810 18:52:23.683940 8169 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:52:23.706406 8169 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:52:23.707726 8169 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:52:23.713539 8169 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 555cf7d9b9714c26baa8aebe456d4e29: Generated new TSK 0
I20260810 18:52:23.714337 8169 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:52:23.720911 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.129:0
--local_ip_for_outbound_sockets=127.18.168.129
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:43707
--builtin_ntp_servers=127.18.168.148:38081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 18:52:23.834225 8180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:23.834492 8180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:23.834563 8180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:23.838547 8180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:23.838620 8180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:23.838707 8180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.129
I20260810 18:52:23.843379 8180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:38081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:43707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.129
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:23.844434 8180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:23.845896 8180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:23.855454 8185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:23.855572 8180 server_base.cc:1034] running on GCE node
W20260810 18:52:23.855517 8186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:23.855454 8188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:23.856128 8180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:23.856813 8180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:23.857980 8180 hybrid_clock.cc:648] HybridClock initialized: now 1786387943857964 us; error 27 us; skew 500 ppm
I20260810 18:52:23.860018 8180 webserver.cc:460] Webserver started at http://127.18.168.129:38915/ using document root <none> and password file <none>
I20260810 18:52:23.860611 8180 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:23.860679 8180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:23.860937 8180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:23.862896 8180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-0/data/instance:
uuid: "0bf50300ab5841318d08209d7c1116f9"
format_stamp: "Formatted at 2026-08-10 18:52:23 on dist-test-slave-cbsf"
server_key: "db96deea21aa8ae99a20b4319052323f"
server_key_iv: "6fa12e3a4877950323d059b86f13cb4a"
server_key_version: "encryptionkey@0"
I20260810 18:52:23.863394 8180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance:
uuid: "0bf50300ab5841318d08209d7c1116f9"
format_stamp: "Formatted at 2026-08-10 18:52:23 on dist-test-slave-cbsf"
server_key: "db96deea21aa8ae99a20b4319052323f"
server_key_iv: "6fa12e3a4877950323d059b86f13cb4a"
server_key_version: "encryptionkey@0"
I20260810 18:52:23.866935 8180 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260810 18:52:23.869282 8195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:23.870450 8180 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:52:23.870602 8180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-0/wal
uuid: "0bf50300ab5841318d08209d7c1116f9"
format_stamp: "Formatted at 2026-08-10 18:52:23 on dist-test-slave-cbsf"
server_key: "db96deea21aa8ae99a20b4319052323f"
server_key_iv: "6fa12e3a4877950323d059b86f13cb4a"
server_key_version: "encryptionkey@0"
I20260810 18:52:23.870716 8180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:23.902832 8180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:23.903641 8180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:23.903872 8180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:23.904633 8180 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:52:23.905738 8180 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:52:23.905819 8180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:23.905901 8180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:52:23.905951 8180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:23.916086 8180 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.129:44391
I20260810 18:52:23.916142 8328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.129:44391 every 8 connection(s)
I20260810 18:52:23.917127 8180 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
I20260810 18:52:23.918501 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 8180
I20260810 18:52:23.918660 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance
I20260810 18:52:23.918975 19106 external_mini_cluster.cc:1428] Setting key f1bcf4c00b80a0c3b00a9e1bba781815
I20260810 18:52:23.921311 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.130:0
--local_ip_for_outbound_sockets=127.18.168.130
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:43707
--builtin_ntp_servers=127.18.168.148:38081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 18:52:23.929121 8158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:23.919235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:59671 (local address 127.18.168.190:43707)
0810 18:52:23.919406 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:23.919412 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:23.920745 (+ 1333us) server_negotiation.cc:407] Connection header received
0810 18:52:23.922788 (+ 2043us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:23.922790 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:23.922838 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:23.922925 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:23.923984 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:23.924967 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:23.926283 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:23.926744 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:23.926890 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:23.928143 (+ 1253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:23.928149 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:23.928153 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:52:23.928170 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:23.928200 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:52:23.928217 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:23.928308 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:23.928620 (+ 312us) server_negotiation.cc:299] Negotiation successful
0810 18:52:23.928778 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 18:52:23.929845 8332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:23.919748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43707 (local address 127.18.168.129:59671)
0810 18:52:23.920556 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:23.920628 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:23.922552 (+ 1924us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:23.923135 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:23.923147 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:23.923262 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:52:23.923827 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:23.923840 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:23.925096 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:23.925098 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:23.926149 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:23.926159 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:23.926898 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:23.926901 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:23.927057 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:23.927796 (+ 739us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:52:23.927798 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:52:23.927851 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:52:23.927933 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:52:23.928368 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:52:23.928388 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:52:23.928394 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 18:52:23.928535 (+ 141us) client_negotiation.cc:240] Negotiation successful
0810 18:52:23.928832 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":165,"threads_started":1}
I20260810 18:52:23.931661 8329 heartbeater.cc:347] Connected to a master server at 127.18.168.190:43707
I20260810 18:52:23.931947 8329 heartbeater.cc:464] Registering TS with master...
I20260810 18:52:23.932729 8329 heartbeater.cc:511] Master 127.18.168.190:43707 requested a full tablet report, sending...
I20260810 18:52:23.934657 8112 ts_manager.cc:194] Registered new tserver with Master: 0bf50300ab5841318d08209d7c1116f9 (127.18.168.129:44391)
I20260810 18:52:23.936525 8112 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.168.129:59671
W20260810 18:52:24.042665 8334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:24.042910 8334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:24.042948 8334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:24.046937 8334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:24.046991 8334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:24.047044 8334 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.130
I20260810 18:52:24.051879 8334 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:38081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:43707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8334
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.130
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:24.052979 8334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:24.054402 8334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:24.064353 8340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:24.064342 8343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:24.064344 8341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:24.064678 8334 server_base.cc:1034] running on GCE node
I20260810 18:52:24.065184 8334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:24.065828 8334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:24.066999 8334 hybrid_clock.cc:648] HybridClock initialized: now 1786387944066969 us; error 42 us; skew 500 ppm
I20260810 18:52:24.068912 8334 webserver.cc:460] Webserver started at http://127.18.168.130:35155/ using document root <none> and password file <none>
I20260810 18:52:24.069514 8334 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:24.069581 8334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:24.069834 8334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:24.071913 8334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-1/data/instance:
uuid: "f085ae73c0844865809eee79ea547a82"
format_stamp: "Formatted at 2026-08-10 18:52:24 on dist-test-slave-cbsf"
server_key: "bd96ce1259833daccb05219b1f6b514c"
server_key_iv: "e620754f7ba75f45377a0250a35b4356"
server_key_version: "encryptionkey@0"
I20260810 18:52:24.072522 8334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance:
uuid: "f085ae73c0844865809eee79ea547a82"
format_stamp: "Formatted at 2026-08-10 18:52:24 on dist-test-slave-cbsf"
server_key: "bd96ce1259833daccb05219b1f6b514c"
server_key_iv: "e620754f7ba75f45377a0250a35b4356"
server_key_version: "encryptionkey@0"
I20260810 18:52:24.076581 8334 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260810 18:52:24.079279 8349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:24.080515 8334 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 18:52:24.080657 8334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-1/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-1/wal
uuid: "f085ae73c0844865809eee79ea547a82"
format_stamp: "Formatted at 2026-08-10 18:52:24 on dist-test-slave-cbsf"
server_key: "bd96ce1259833daccb05219b1f6b514c"
server_key_iv: "e620754f7ba75f45377a0250a35b4356"
server_key_version: "encryptionkey@0"
I20260810 18:52:24.080809 8334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:24.107558 8334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:24.108261 8334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:24.108474 8334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:24.109117 8334 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:52:24.110129 8334 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:52:24.110208 8334 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:24.110283 8334 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:52:24.110325 8334 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:24.119760 8334 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.130:33949
I20260810 18:52:24.119812 8478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.130:33949 every 8 connection(s)
I20260810 18:52:24.120682 8334 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
I20260810 18:52:24.129948 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 8334
I20260810 18:52:24.130096 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance
I20260810 18:52:24.130131 8158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:24.122570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:47549 (local address 127.18.168.190:43707)
0810 18:52:24.122855 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:24.122860 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:24.123946 (+ 1086us) server_negotiation.cc:407] Connection header received
0810 18:52:24.125476 (+ 1530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:24.125479 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:24.125526 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:24.125601 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:24.126552 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:24.127099 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:24.128211 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:24.128416 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:24.128491 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:24.129596 (+ 1105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:24.129599 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:24.129601 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:52:24.129612 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:24.129631 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:52:24.129656 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:24.129711 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:24.129899 (+ 188us) server_negotiation.cc:299] Negotiation successful
0810 18:52:24.129949 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164}
I20260810 18:52:24.130440 19106 external_mini_cluster.cc:1428] Setting key 97bce43873a91786e12f0bb135417b66
I20260810 18:52:24.131148 8481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:24.123098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43707 (local address 127.18.168.130:47549)
0810 18:52:24.123785 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:24.123825 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:24.125271 (+ 1446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:24.125759 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:24.125766 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:24.125844 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:52:24.126396 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:24.126409 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:24.127213 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:24.127215 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:24.128099 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:24.128109 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:24.128542 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:24.128544 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:24.128751 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:24.129345 (+ 594us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:52:24.129347 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:52:24.129385 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:52:24.129446 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:52:24.129738 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:52:24.129749 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:52:24.129756 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 18:52:24.129896 (+ 140us) client_negotiation.cc:240] Negotiation successful
0810 18:52:24.130229 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":127,"threads_started":1}
I20260810 18:52:24.132258 8479 heartbeater.cc:347] Connected to a master server at 127.18.168.190:43707
I20260810 18:52:24.132508 8479 heartbeater.cc:464] Registering TS with master...
I20260810 18:52:24.132719 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.131:0
--local_ip_for_outbound_sockets=127.18.168.131
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:43707
--builtin_ntp_servers=127.18.168.148:38081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 18:52:24.133157 8479 heartbeater.cc:511] Master 127.18.168.190:43707 requested a full tablet report, sending...
I20260810 18:52:24.134266 8111 ts_manager.cc:194] Registered new tserver with Master: f085ae73c0844865809eee79ea547a82 (127.18.168.130:33949)
I20260810 18:52:24.134864 8111 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.168.130:47549
W20260810 18:52:24.247655 8482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:24.247897 8482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:24.247961 8482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:24.252084 8482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:24.252153 8482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:24.252205 8482 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.131
I20260810 18:52:24.256867 8482 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:38081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:43707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.131
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:24.257918 8482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:24.259334 8482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:24.269101 8487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:24.269218 8482 server_base.cc:1034] running on GCE node
W20260810 18:52:24.269085 8488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:24.269085 8492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:24.269815 8482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:24.270457 8482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:24.271644 8482 hybrid_clock.cc:648] HybridClock initialized: now 1786387944271612 us; error 43 us; skew 500 ppm
I20260810 18:52:24.273684 8482 webserver.cc:460] Webserver started at http://127.18.168.131:46663/ using document root <none> and password file <none>
I20260810 18:52:24.274223 8482 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:24.274281 8482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:24.274560 8482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:24.276763 8482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-2/data/instance:
uuid: "064b95540ff34ebbad14ad3ce366ada0"
format_stamp: "Formatted at 2026-08-10 18:52:24 on dist-test-slave-cbsf"
server_key: "dcd4003da313e72e918401cae492ed0c"
server_key_iv: "2238f76489b5877c25e16e36b3ffee43"
server_key_version: "encryptionkey@0"
I20260810 18:52:24.277307 8482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance:
uuid: "064b95540ff34ebbad14ad3ce366ada0"
format_stamp: "Formatted at 2026-08-10 18:52:24 on dist-test-slave-cbsf"
server_key: "dcd4003da313e72e918401cae492ed0c"
server_key_iv: "2238f76489b5877c25e16e36b3ffee43"
server_key_version: "encryptionkey@0"
I20260810 18:52:24.280823 8482 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 18:52:24.283093 8499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:24.284315 8482 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:52:24.284435 8482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-2/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-2/wal
uuid: "064b95540ff34ebbad14ad3ce366ada0"
format_stamp: "Formatted at 2026-08-10 18:52:24 on dist-test-slave-cbsf"
server_key: "dcd4003da313e72e918401cae492ed0c"
server_key_iv: "2238f76489b5877c25e16e36b3ffee43"
server_key_version: "encryptionkey@0"
I20260810 18:52:24.284531 8482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:24.295878 8482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:24.296408 8482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:24.296576 8482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:24.297147 8482 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:52:24.298008 8482 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:52:24.298058 8482 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:24.298094 8482 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:52:24.298115 8482 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:24.306576 8482 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.131:45599
I20260810 18:52:24.306665 8627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.131:45599 every 8 connection(s)
I20260810 18:52:24.307456 8482 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
I20260810 18:52:24.309966 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 8482
I20260810 18:52:24.310093 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance
I20260810 18:52:24.310396 19106 external_mini_cluster.cc:1428] Setting key f6fe2a178939cd04bbae2be0ceb8c726
I20260810 18:52:24.316928 8158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:24.309294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:54695 (local address 127.18.168.190:43707)
0810 18:52:24.309437 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:24.309441 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:24.310627 (+ 1186us) server_negotiation.cc:407] Connection header received
0810 18:52:24.312361 (+ 1734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:24.312363 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:24.312427 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:24.312504 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:24.313468 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:24.314105 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:24.315220 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:24.315433 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:24.315511 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:24.316471 (+ 960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:24.316474 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:24.316476 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:52:24.316487 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:24.316505 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:52:24.316513 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:24.316576 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:24.316761 (+ 185us) server_negotiation.cc:299] Negotiation successful
0810 18:52:24.316824 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 18:52:24.317770 8630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:24.309662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43707 (local address 127.18.168.131:54695)
0810 18:52:24.310464 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:24.310520 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:24.312132 (+ 1612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:24.312770 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:24.312779 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:24.312866 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:52:24.313338 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:24.313350 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:24.314231 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:24.314234 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:24.315114 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:24.315124 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:24.315575 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:24.315578 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:24.315715 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:24.316246 (+ 531us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:52:24.316248 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:52:24.316285 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:52:24.316343 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:52:24.316590 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:52:24.316597 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:52:24.316601 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 18:52:24.316740 (+ 139us) client_negotiation.cc:240] Negotiation successful
0810 18:52:24.316984 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":112,"threads_started":1}
I20260810 18:52:24.318761 8628 heartbeater.cc:347] Connected to a master server at 127.18.168.190:43707
I20260810 18:52:24.318991 8628 heartbeater.cc:464] Registering TS with master...
I20260810 18:52:24.319481 8628 heartbeater.cc:511] Master 127.18.168.190:43707 requested a full tablet report, sending...
I20260810 18:52:24.320405 8111 ts_manager.cc:194] Registered new tserver with Master: 064b95540ff34ebbad14ad3ce366ada0 (127.18.168.131:45599)
I20260810 18:52:24.321064 8111 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.168.131:54695
I20260810 18:52:24.324295 19106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:52:24.329813 8158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:24.326400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52012 (local address 127.18.168.190:43707)
0810 18:52:24.326508 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:24.326511 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:24.326710 (+ 199us) server_negotiation.cc:407] Connection header received
0810 18:52:24.326805 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:24.326807 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:24.326840 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:24.326904 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:24.327158 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:24.327624 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:24.328398 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:24.328598 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:24.328669 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:24.329443 (+ 774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:24.329446 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:24.329448 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:52:24.329459 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:24.329476 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:52:24.329483 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:24.329581 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:24.329693 (+ 112us) server_negotiation.cc:299] Negotiation successful
0810 18:52:24.329723 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 18:52:24.332600 8111 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52012:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:52:24.335575 8111 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:52:24.349310 8640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:24.343875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.131:45599 (local address 127.0.0.1:52492)
0810 18:52:24.344869 (+ 994us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:24.344882 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:24.344966 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:24.345462 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:24.345467 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:24.345490 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:24.345635 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:24.345642 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:24.346686 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:24.346692 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:24.347907 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:24.347917 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:24.348750 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:24.348754 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:24.348880 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:24.348932 (+ 52us) client_negotiation.cc:769] Sending connection context
0810 18:52:24.349057 (+ 125us) client_negotiation.cc:240] Negotiation successful
0810 18:52:24.349170 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":932,"mutex_wait_us":6,"thread_start_us":103,"threads_started":1}
I20260810 18:52:24.349781 8644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:24.344295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52492 (local address 127.18.168.131:45599)
0810 18:52:24.344830 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:24.344844 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:24.345060 (+ 216us) server_negotiation.cc:407] Connection header received
0810 18:52:24.345145 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:24.345150 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:24.345313 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:24.345467 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:24.345760 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:24.346572 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:24.348046 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:24.348593 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:24.348752 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:24.348936 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:24.349500 (+ 564us) server_negotiation.cc:299] Negotiation successful
0810 18:52:24.349605 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":125,"threads_started":1}
I20260810 18:52:24.350278 8639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:24.343521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:44391 (local address 127.0.0.1:42628)
0810 18:52:24.344785 (+ 1264us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:24.344800 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:24.345056 (+ 256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:24.345630 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:24.345632 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:24.345648 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:24.345753 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:24.345758 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:24.347235 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:24.347239 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:24.348919 (+ 1680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:24.348925 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:24.349931 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:24.349934 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:24.350029 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:24.350074 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 18:52:24.350134 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 18:52:24.350186 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"mutex_wait_us":362,"thread_start_us":89,"threads_started":1}
I20260810 18:52:24.350559 8641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:24.343660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42628 (local address 127.18.168.129:44391)
0810 18:52:24.344092 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:24.344106 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:24.344890 (+ 784us) server_negotiation.cc:407] Connection header received
0810 18:52:24.345209 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:24.345214 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:24.345376 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:24.345531 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:24.345874 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:24.347095 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:24.349027 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:24.349796 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:24.349930 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:24.350106 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:24.350158 (+ 52us) server_negotiation.cc:299] Negotiation successful
0810 18:52:24.350243 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":142,"threads_started":1}
I20260810 18:52:24.352008 8642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:24.343606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:33949 (local address 127.0.0.1:45764)
0810 18:52:24.344321 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:24.344368 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:24.344483 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:24.345678 (+ 1195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:24.345683 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:24.345727 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:24.345823 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:24.345829 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:24.347235 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:24.347239 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:24.348911 (+ 1672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:24.348923 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:24.351614 (+ 2691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:24.351618 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:24.351710 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:24.351732 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 18:52:24.351805 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 18:52:24.351872 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"mutex_wait_us":303,"thread_start_us":103,"threads_started":1}
I20260810 18:52:24.352495 8643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:24.344296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45764 (local address 127.18.168.130:33949)
0810 18:52:24.345059 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:24.345068 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:24.345087 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:52:24.345185 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:24.345190 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:24.345376 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:24.345529 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:24.345932 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:24.347095 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:24.350711 (+ 3616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:24.351244 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:24.351390 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:24.351546 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:24.352230 (+ 684us) server_negotiation.cc:299] Negotiation successful
0810 18:52:24.352333 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":123,"threads_started":1}
I20260810 18:52:24.352910 8554 tablet_service.cc:1467] Processing CreateTablet for tablet 2a7ab6fe3fe941588c44f0f116a0812f (DEFAULT_TABLE table=test-table [id=d367083bc7534a07a79fe5ab5a5fb2c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:52:24.353343 8250 tablet_service.cc:1467] Processing CreateTablet for tablet 2a7ab6fe3fe941588c44f0f116a0812f (DEFAULT_TABLE table=test-table [id=d367083bc7534a07a79fe5ab5a5fb2c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:52:24.354050 8554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a7ab6fe3fe941588c44f0f116a0812f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:24.354302 8250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a7ab6fe3fe941588c44f0f116a0812f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:24.355127 8404 tablet_service.cc:1467] Processing CreateTablet for tablet 2a7ab6fe3fe941588c44f0f116a0812f (DEFAULT_TABLE table=test-table [id=d367083bc7534a07a79fe5ab5a5fb2c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:52:24.356087 8404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a7ab6fe3fe941588c44f0f116a0812f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:24.361122 8645 tablet_bootstrap.cc:492] T 2a7ab6fe3fe941588c44f0f116a0812f P 0bf50300ab5841318d08209d7c1116f9: Bootstrap starting.
I20260810 18:52:24.361795 8646 tablet_bootstrap.cc:492] T 2a7ab6fe3fe941588c44f0f116a0812f P 064b95540ff34ebbad14ad3ce366ada0: Bootstrap starting.
I20260810 18:52:24.362800 8647 tablet_bootstrap.cc:492] T 2a7ab6fe3fe941588c44f0f116a0812f P f085ae73c0844865809eee79ea547a82: Bootstrap starting.
I20260810 18:52:24.363384 8645 tablet_bootstrap.cc:654] T 2a7ab6fe3fe941588c44f0f116a0812f P 0bf50300ab5841318d08209d7c1116f9: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:24.363849 8646 tablet_bootstrap.cc:654] T 2a7ab6fe3fe941588c44f0f116a0812f P 064b95540ff34ebbad14ad3ce366ada0: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:24.364285 8645 log.cc:824] T 2a7ab6fe3fe941588c44f0f116a0812f P 0bf50300ab5841318d08209d7c1116f9: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:24.364563 8646 log.cc:824] T 2a7ab6fe3fe941588c44f0f116a0812f P 064b95540ff34ebbad14ad3ce366ada0: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:24.364895 8647 tablet_bootstrap.cc:654] T 2a7ab6fe3fe941588c44f0f116a0812f P f085ae73c0844865809eee79ea547a82: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:24.365710 8647 log.cc:824] T 2a7ab6fe3fe941588c44f0f116a0812f P f085ae73c0844865809eee79ea547a82: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:24.366529 8646 tablet_bootstrap.cc:492] T 2a7ab6fe3fe941588c44f0f116a0812f P 064b95540ff34ebbad14ad3ce366ada0: No bootstrap required, opened a new log
I20260810 18:52:24.366555 8645 tablet_bootstrap.cc:492] T 2a7ab6fe3fe941588c44f0f116a0812f P 0bf50300ab5841318d08209d7c1116f9: No bootstrap required, opened a new log
I20260810 18:52:24.366739 8646 ts_tablet_manager.cc:1397] T 2a7ab6fe3fe941588c44f0f116a0812f P 064b95540ff34ebbad14ad3ce366ada0: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260810 18:52:24.366739 8645 ts_tablet_manager.cc:1397] T 2a7ab6fe3fe941588c44f0f116a0812f P 0bf50300ab5841318d08209d7c1116f9: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 18:52:24.368130 8647 tablet_bootstrap.cc:492] T 2a7ab6fe3fe941588c44f0f116a0812f P f085ae73c0844865809eee79ea547a82: No bootstrap required, opened a new log
I20260810 18:52:24.368310 8647 ts_tablet_manager.cc:1397] T 2a7ab6fe3fe941588c44f0f116a0812f P f085ae73c0844865809eee79ea547a82: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260810 18:52:24.370131 8646 raft_consensus.cc:348] T 2a7ab6fe3fe941588c44f0f116a0812f P 064b95540ff34ebbad14ad3ce366ada0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf50300ab5841318d08209d7c1116f9" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44391 } } peers { permanent_uuid: "f085ae73c0844865809eee79ea547a82" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 33949 } } peers { permanent_uuid: "064b95540ff34ebbad14ad3ce366ada0" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45599 } }
I20260810 18:52:24.370358 8646 raft_consensus.cc:374] T 2a7ab6fe3fe941588c44f0f116a0812f P 064b95540ff34ebbad14ad3ce366ada0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:24.370410 8646 raft_consensus.cc:729] T 2a7ab6fe3fe941588c44f0f116a0812f P 064b95540ff34ebbad14ad3ce366ada0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 064b95540ff34ebbad14ad3ce366ada0, State: Initialized, Role: FOLLOWER
I20260810 18:52:24.370863 8646 consensus_queue.cc:260] T 2a7ab6fe3fe941588c44f0f116a0812f P 064b95540ff34ebbad14ad3ce366ada0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf50300ab5841318d08209d7c1116f9" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44391 } } peers { permanent_uuid: "f085ae73c0844865809eee79ea547a82" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 33949 } } peers { permanent_uuid: "064b95540ff34ebbad14ad3ce366ada0" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45599 } }
I20260810 18:52:24.371033 8645 raft_consensus.cc:348] T 2a7ab6fe3fe941588c44f0f116a0812f P 0bf50300ab5841318d08209d7c1116f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf50300ab5841318d08209d7c1116f9" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44391 } } peers { permanent_uuid: "f085ae73c0844865809eee79ea547a82" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 33949 } } peers { permanent_uuid: "064b95540ff34ebbad14ad3ce366ada0" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45599 } }
I20260810 18:52:24.371272 8645 raft_consensus.cc:374] T 2a7ab6fe3fe941588c44f0f116a0812f P 0bf50300ab5841318d08209d7c1116f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:24.371344 8645 raft_consensus.cc:729] T 2a7ab6fe3fe941588c44f0f116a0812f P 0bf50300ab5841318d08209d7c1116f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bf50300ab5841318d08209d7c1116f9, State: Initialized, Role: FOLLOWER
I20260810 18:52:24.371614 8628 heartbeater.cc:503] Master 127.18.168.190:43707 was elected leader, sending a full tablet report...
I20260810 18:52:24.371938 8646 ts_tablet_manager.cc:1428] T 2a7ab6fe3fe941588c44f0f116a0812f P 064b95540ff34ebbad14ad3ce366ada0: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260810 18:52:24.371877 8645 consensus_queue.cc:260] T 2a7ab6fe3fe941588c44f0f116a0812f P 0bf50300ab5841318d08209d7c1116f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf50300ab5841318d08209d7c1116f9" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44391 } } peers { permanent_uuid: "f085ae73c0844865809eee79ea547a82" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 33949 } } peers { permanent_uuid: "064b95540ff34ebbad14ad3ce366ada0" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45599 } }
I20260810 18:52:24.372100 8647 raft_consensus.cc:348] T 2a7ab6fe3fe941588c44f0f116a0812f P f085ae73c0844865809eee79ea547a82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf50300ab5841318d08209d7c1116f9" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44391 } } peers { permanent_uuid: "f085ae73c0844865809eee79ea547a82" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 33949 } } peers { permanent_uuid: "064b95540ff34ebbad14ad3ce366ada0" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45599 } }
I20260810 18:52:24.372324 8647 raft_consensus.cc:374] T 2a7ab6fe3fe941588c44f0f116a0812f P f085ae73c0844865809eee79ea547a82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:24.372370 8647 raft_consensus.cc:729] T 2a7ab6fe3fe941588c44f0f116a0812f P f085ae73c0844865809eee79ea547a82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f085ae73c0844865809eee79ea547a82, State: Initialized, Role: FOLLOWER
I20260810 18:52:24.372787 8645 ts_tablet_manager.cc:1428] T 2a7ab6fe3fe941588c44f0f116a0812f P 0bf50300ab5841318d08209d7c1116f9: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20260810 18:52:24.372840 8647 consensus_queue.cc:260] T 2a7ab6fe3fe941588c44f0f116a0812f P f085ae73c0844865809eee79ea547a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf50300ab5841318d08209d7c1116f9" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44391 } } peers { permanent_uuid: "f085ae73c0844865809eee79ea547a82" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 33949 } } peers { permanent_uuid: "064b95540ff34ebbad14ad3ce366ada0" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45599 } }
I20260810 18:52:24.373747 8479 heartbeater.cc:503] Master 127.18.168.190:43707 was elected leader, sending a full tablet report...
I20260810 18:52:24.373832 8329 heartbeater.cc:503] Master 127.18.168.190:43707 was elected leader, sending a full tablet report...
I20260810 18:52:24.374358 8647 ts_tablet_manager.cc:1428] T 2a7ab6fe3fe941588c44f0f116a0812f P f085ae73c0844865809eee79ea547a82: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260810 18:52:24.395174 8657 raft_consensus.cc:482] T 2a7ab6fe3fe941588c44f0f116a0812f P 0bf50300ab5841318d08209d7c1116f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:52:24.395380 8657 raft_consensus.cc:504] T 2a7ab6fe3fe941588c44f0f116a0812f P 0bf50300ab5841318d08209d7c1116f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf50300ab5841318d08209d7c1116f9" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44391 } } peers { permanent_uuid: "f085ae73c0844865809eee79ea547a82" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 33949 } } peers { permanent_uuid: "064b95540ff34ebbad14ad3ce366ada0" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45599 } }
I20260810 18:52:24.396508 8657 leader_election.cc:288] T 2a7ab6fe3fe941588c44f0f116a0812f P 0bf50300ab5841318d08209d7c1116f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f085ae73c0844865809eee79ea547a82 (127.18.168.130:33949), 064b95540ff34ebbad14ad3ce366ada0 (127.18.168.131:45599)
I20260810 18:52:24.401391 8644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:24.396919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:33743 (local address 127.18.168.131:45599)
0810 18:52:24.397086 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:24.397091 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:24.397391 (+ 300us) server_negotiation.cc:407] Connection header received
0810 18:52:24.397528 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:24.397531 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:24.397591 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:24.397650 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:24.397951 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:24.398657 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:24.400088 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:24.400715 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:24.400839 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:24.400892 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:24.401189 (+ 297us) server_negotiation.cc:299] Negotiation successful
0810 18:52:24.401260 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 18:52:24.401391 8643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:24.397069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:36497 (local address 127.18.168.130:33949)
0810 18:52:24.397215 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:24.397219 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:24.397235 (+ 16us) server_negotiation.cc:407] Connection header received
0810 18:52:24.397292 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:24.397294 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:24.397332 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:24.397426 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:24.397743 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:24.398514 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:24.400088 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:24.400723 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:24.400839 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:24.400892 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:24.401189 (+ 297us) server_negotiation.cc:299] Negotiation successful
0810 18:52:24.401260 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 18:52:24.401407 8332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:24.396797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:33949 (local address 127.18.168.129:36497)
0810 18:52:24.397038 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:24.397054 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:24.397144 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:24.397457 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:24.397462 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:24.397480 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:24.397605 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:24.397610 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:24.398621 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:24.398624 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:24.399954 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:24.399966 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:24.400844 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:24.400848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:24.401039 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:24.401079 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 18:52:24.401160 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 18:52:24.401264 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":89}
I20260810 18:52:24.401417 8659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:24.396797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.131:45599 (local address 127.18.168.129:33743)
0810 18:52:24.397286 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:24.397298 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:24.397390 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:24.397698 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:24.397701 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:24.397720 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:24.397833 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:24.397837 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:24.398755 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:24.398758 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:24.399974 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:24.399980 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:24.400846 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:24.400849 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:24.401013 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:24.401090 (+ 77us) client_negotiation.cc:769] Sending connection context
0810 18:52:24.401160 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 18:52:24.401264 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"mutex_wait_us":213,"thread_start_us":152,"threads_started":1}
I20260810 18:52:24.402211 8425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a7ab6fe3fe941588c44f0f116a0812f" candidate_uuid: "0bf50300ab5841318d08209d7c1116f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f085ae73c0844865809eee79ea547a82" is_pre_election: true
I20260810 18:52:24.402216 8577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a7ab6fe3fe941588c44f0f116a0812f" candidate_uuid: "0bf50300ab5841318d08209d7c1116f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "064b95540ff34ebbad14ad3ce366ada0" is_pre_election: true
I20260810 18:52:24.402505 8425 raft_consensus.cc:2454] T 2a7ab6fe3fe941588c44f0f116a0812f P f085ae73c0844865809eee79ea547a82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bf50300ab5841318d08209d7c1116f9 in term 0.
I20260810 18:52:24.402527 8577 raft_consensus.cc:2454] T 2a7ab6fe3fe941588c44f0f116a0812f P 064b95540ff34ebbad14ad3ce366ada0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bf50300ab5841318d08209d7c1116f9 in term 0.
I20260810 18:52:24.403018 8199 leader_election.cc:302] T 2a7ab6fe3fe941588c44f0f116a0812f P 0bf50300ab5841318d08209d7c1116f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bf50300ab5841318d08209d7c1116f9, f085ae73c0844865809eee79ea547a82; no voters:
I20260810 18:52:24.403304 8657 raft_consensus.cc:2781] T 2a7ab6fe3fe941588c44f0f116a0812f P 0bf50300ab5841318d08209d7c1116f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:52:24.403395 8657 raft_consensus.cc:482] T 2a7ab6fe3fe941588c44f0f116a0812f P 0bf50300ab5841318d08209d7c1116f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:52:24.403440 8657 raft_consensus.cc:3037] T 2a7ab6fe3fe941588c44f0f116a0812f P 0bf50300ab5841318d08209d7c1116f9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:24.404613 8657 raft_consensus.cc:504] T 2a7ab6fe3fe941588c44f0f116a0812f P 0bf50300ab5841318d08209d7c1116f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf50300ab5841318d08209d7c1116f9" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44391 } } peers { permanent_uuid: "f085ae73c0844865809eee79ea547a82" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 33949 } } peers { permanent_uuid: "064b95540ff34ebbad14ad3ce366ada0" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45599 } }
I20260810 18:52:24.405071 8657 leader_election.cc:288] T 2a7ab6fe3fe941588c44f0f116a0812f P 0bf50300ab5841318d08209d7c1116f9 [CANDIDATE]: Term 1 election: Requested vote from peers f085ae73c0844865809eee79ea547a82 (127.18.168.130:33949), 064b95540ff34ebbad14ad3ce366ada0 (127.18.168.131:45599)
I20260810 18:52:24.405383 8425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a7ab6fe3fe941588c44f0f116a0812f" candidate_uuid: "0bf50300ab5841318d08209d7c1116f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f085ae73c0844865809eee79ea547a82"
I20260810 18:52:24.405426 8577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a7ab6fe3fe941588c44f0f116a0812f" candidate_uuid: "0bf50300ab5841318d08209d7c1116f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "064b95540ff34ebbad14ad3ce366ada0"
I20260810 18:52:24.405490 8425 raft_consensus.cc:3037] T 2a7ab6fe3fe941588c44f0f116a0812f P f085ae73c0844865809eee79ea547a82 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:24.405576 8577 raft_consensus.cc:3037] T 2a7ab6fe3fe941588c44f0f116a0812f P 064b95540ff34ebbad14ad3ce366ada0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:24.406442 8425 raft_consensus.cc:2454] T 2a7ab6fe3fe941588c44f0f116a0812f P f085ae73c0844865809eee79ea547a82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bf50300ab5841318d08209d7c1116f9 in term 1.
I20260810 18:52:24.406672 8577 raft_consensus.cc:2454] T 2a7ab6fe3fe941588c44f0f116a0812f P 064b95540ff34ebbad14ad3ce366ada0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bf50300ab5841318d08209d7c1116f9 in term 1.
I20260810 18:52:24.406708 8199 leader_election.cc:302] T 2a7ab6fe3fe941588c44f0f116a0812f P 0bf50300ab5841318d08209d7c1116f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bf50300ab5841318d08209d7c1116f9, f085ae73c0844865809eee79ea547a82; no voters:
I20260810 18:52:24.406862 8657 raft_consensus.cc:2781] T 2a7ab6fe3fe941588c44f0f116a0812f P 0bf50300ab5841318d08209d7c1116f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:52:24.407078 8657 raft_consensus.cc:686] T 2a7ab6fe3fe941588c44f0f116a0812f P 0bf50300ab5841318d08209d7c1116f9 [term 1 LEADER]: Becoming Leader. State: Replica: 0bf50300ab5841318d08209d7c1116f9, State: Running, Role: LEADER
I20260810 18:52:24.407441 8657 consensus_queue.cc:237] T 2a7ab6fe3fe941588c44f0f116a0812f P 0bf50300ab5841318d08209d7c1116f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf50300ab5841318d08209d7c1116f9" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44391 } } peers { permanent_uuid: "f085ae73c0844865809eee79ea547a82" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 33949 } } peers { permanent_uuid: "064b95540ff34ebbad14ad3ce366ada0" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45599 } }
I20260810 18:52:24.410537 8112 catalog_manager.cc:5495] T 2a7ab6fe3fe941588c44f0f116a0812f P 0bf50300ab5841318d08209d7c1116f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0bf50300ab5841318d08209d7c1116f9 (127.18.168.129). New cstate: current_term: 1 leader_uuid: "0bf50300ab5841318d08209d7c1116f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf50300ab5841318d08209d7c1116f9" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 44391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f085ae73c0844865809eee79ea547a82" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 33949 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "064b95540ff34ebbad14ad3ce366ada0" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45599 } health_report { overall_health: UNKNOWN } } }
W20260810 18:52:24.418373 8330 tablet.cc:2369] T 2a7ab6fe3fe941588c44f0f116a0812f P 0bf50300ab5841318d08209d7c1116f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:52:24.428383 8641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:24.424673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42640 (local address 127.18.168.129:44391)
0810 18:52:24.424849 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:24.424852 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:24.424868 (+ 16us) server_negotiation.cc:407] Connection header received
0810 18:52:24.425041 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:24.425043 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:24.425112 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:24.425219 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:52:24.425466 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:24.426070 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:24.426961 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:24.427171 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:24.427251 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:24.427552 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:52:24.427924 (+ 372us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:52:24.428035 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:24.428187 (+ 152us) server_negotiation.cc:299] Negotiation successful
0810 18:52:24.428229 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260810 18:52:24.433900 8425 raft_consensus.cc:1261] T 2a7ab6fe3fe941588c44f0f116a0812f P f085ae73c0844865809eee79ea547a82 [term 1 FOLLOWER]: Refusing update from remote peer 0bf50300ab5841318d08209d7c1116f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:52:24.433900 8577 raft_consensus.cc:1261] T 2a7ab6fe3fe941588c44f0f116a0812f P 064b95540ff34ebbad14ad3ce366ada0 [term 1 FOLLOWER]: Refusing update from remote peer 0bf50300ab5841318d08209d7c1116f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:52:24.434616 8660 consensus_queue.cc:1035] T 2a7ab6fe3fe941588c44f0f116a0812f P 0bf50300ab5841318d08209d7c1116f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "064b95540ff34ebbad14ad3ce366ada0" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 45599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:52:24.434834 8657 consensus_queue.cc:1035] T 2a7ab6fe3fe941588c44f0f116a0812f P 0bf50300ab5841318d08209d7c1116f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f085ae73c0844865809eee79ea547a82" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 33949 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:52:24.441987 8666 mvcc.cc:205] Tried to move back new op lower bound from 7317045020394270720 to 7317045020295696384. Current Snapshot: MvccSnapshot[applied={T|T < 7317045020394270720}]
I20260810 18:52:24.442204 8664 mvcc.cc:205] Tried to move back new op lower bound from 7317045020394270720 to 7317045020295696384. Current Snapshot: MvccSnapshot[applied={T|T < 7317045020394270720}]
I20260810 18:52:24.443488 8665 mvcc.cc:205] Tried to move back new op lower bound from 7317045020394270720 to 7317045020295696384. Current Snapshot: MvccSnapshot[applied={T|T < 7317045020394270720}]
I20260810 18:52:24.450966 8112 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52012:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:52:24.451138 8112 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52012:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:52:24.453814 8112 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 555cf7d9b9714c26baa8aebe456d4e29: Sending DeleteTablet for 3 replicas of tablet 2a7ab6fe3fe941588c44f0f116a0812f
I20260810 18:52:24.454552 8250 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a7ab6fe3fe941588c44f0f116a0812f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:52:24 UTC) from {username='slave'} at 127.0.0.1:42628
I20260810 18:52:24.454632 8404 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a7ab6fe3fe941588c44f0f116a0812f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:52:24 UTC) from {username='slave'} at 127.0.0.1:45764
I20260810 18:52:24.455063 8674 tablet_replica.cc:330] T 2a7ab6fe3fe941588c44f0f116a0812f P f085ae73c0844865809eee79ea547a82: stopping tablet replica
I20260810 18:52:24.455055 8554 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a7ab6fe3fe941588c44f0f116a0812f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:52:24 UTC) from {username='slave'} at 127.0.0.1:52492
I20260810 18:52:24.455394 8674 raft_consensus.cc:2229] T 2a7ab6fe3fe941588c44f0f116a0812f P f085ae73c0844865809eee79ea547a82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:52:24.455565 8675 tablet_replica.cc:330] T 2a7ab6fe3fe941588c44f0f116a0812f P 064b95540ff34ebbad14ad3ce366ada0: stopping tablet replica
I20260810 18:52:24.455731 8674 raft_consensus.cc:2258] T 2a7ab6fe3fe941588c44f0f116a0812f P f085ae73c0844865809eee79ea547a82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:52:24.455947 8675 raft_consensus.cc:2229] T 2a7ab6fe3fe941588c44f0f116a0812f P 064b95540ff34ebbad14ad3ce366ada0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:52:24.456296 8675 raft_consensus.cc:2258] T 2a7ab6fe3fe941588c44f0f116a0812f P 064b95540ff34ebbad14ad3ce366ada0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:52:24.456477 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 8180
I20260810 18:52:24.458623 8674 ts_tablet_manager.cc:1905] T 2a7ab6fe3fe941588c44f0f116a0812f P f085ae73c0844865809eee79ea547a82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:52:24.459801 8675 ts_tablet_manager.cc:1905] T 2a7ab6fe3fe941588c44f0f116a0812f P 064b95540ff34ebbad14ad3ce366ada0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:52:24.463809 8675 ts_tablet_manager.cc:1918] T 2a7ab6fe3fe941588c44f0f116a0812f P 064b95540ff34ebbad14ad3ce366ada0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:52:24.464018 8675 log.cc:1196] T 2a7ab6fe3fe941588c44f0f116a0812f P 064b95540ff34ebbad14ad3ce366ada0: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-2/wal/wals/2a7ab6fe3fe941588c44f0f116a0812f
I20260810 18:52:24.464517 8675 ts_tablet_manager.cc:1939] T 2a7ab6fe3fe941588c44f0f116a0812f P 064b95540ff34ebbad14ad3ce366ada0: Deleting consensus metadata
I20260810 18:52:24.466327 8096 catalog_manager.cc:4841] TS 064b95540ff34ebbad14ad3ce366ada0 (127.18.168.131:45599): tablet 2a7ab6fe3fe941588c44f0f116a0812f (table test-table [id=d367083bc7534a07a79fe5ab5a5fb2c0]) successfully deleted
I20260810 18:52:24.467746 8674 ts_tablet_manager.cc:1918] T 2a7ab6fe3fe941588c44f0f116a0812f P f085ae73c0844865809eee79ea547a82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:52:24.467834 8674 log.cc:1196] T 2a7ab6fe3fe941588c44f0f116a0812f P f085ae73c0844865809eee79ea547a82: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387687857264-19106-0/minicluster-data/ts-1/wal/wals/2a7ab6fe3fe941588c44f0f116a0812f
I20260810 18:52:24.468154 8674 ts_tablet_manager.cc:1939] T 2a7ab6fe3fe941588c44f0f116a0812f P f085ae73c0844865809eee79ea547a82: Deleting consensus metadata
I20260810 18:52:24.469456 8094 catalog_manager.cc:4841] TS f085ae73c0844865809eee79ea547a82 (127.18.168.130:33949): tablet 2a7ab6fe3fe941588c44f0f116a0812f (table test-table [id=d367083bc7534a07a79fe5ab5a5fb2c0]) successfully deleted
W20260810 18:52:24.475539 8095 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.168.129:44391 (error 108)
I20260810 18:52:24.476359 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 8334
I20260810 18:52:24.476675 8642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:24.476316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:44391 (local address 127.0.0.1:42652)
0810 18:52:24.476493 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:24.476570 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.129:44391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 18:52:24.476969 8095 catalog_manager.cc:4584] TS 0bf50300ab5841318d08209d7c1116f9 (127.18.168.129:44391): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2a7ab6fe3fe941588c44f0f116a0812f: Network error: Client connection negotiation failed: client connection to 127.18.168.129:44391: connect: Connection refused (error 111)
I20260810 18:52:24.484985 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 8482
I20260810 18:52:24.493912 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 8075
2026-08-10T18:52:24Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1058 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:52:24 dist-test-slave-cbsf krb5kdc[8679](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:52:24 dist-test-slave-cbsf krb5kdc[8679](info): set up 2 sockets
Aug 10 18:52:24 dist-test-slave-cbsf krb5kdc[8679](info): commencing operation
krb5kdc: starting...
W20260810 18:52:25.606899 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:52:25 dist-test-slave-cbsf krb5kdc[8679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:52:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:52:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.168.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
I20260810 18:52:25.734081 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:35543
--webserver_interface=127.18.168.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:36543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:35543
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:52:25.848294 8694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:25.848565 8694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:25.848608 8694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:25.852598 8694 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:52:25.852653 8694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:25.852672 8694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:25.852703 8694 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:52:25.852720 8694 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:52:25.857807 8694 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:36543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:35543
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:35543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:25.858768 8694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:25.859778 8694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:25.868497 8702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:25.868551 8700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:25.868666 8694 server_base.cc:1034] running on GCE node
W20260810 18:52:25.868500 8699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:25.869300 8694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:25.870425 8694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:25.871620 8694 hybrid_clock.cc:648] HybridClock initialized: now 1786387945871613 us; error 29 us; skew 500 ppm
Aug 10 18:52:25 dist-test-slave-cbsf krb5kdc[8679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387945, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:25.874346 8694 init.cc:377] Logged in from keytab as kudu/127.18.168.190@KRBTEST.COM (short username kudu)
I20260810 18:52:25.875241 8694 webserver.cc:460] Webserver started at http://127.18.168.190:35373/ using document root <none> and password file <none>
I20260810 18:52:25.875799 8694 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:25.875890 8694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:25.876128 8694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:25.878203 8694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "dba3fe05861c4bbca11f744edd711f6a"
format_stamp: "Formatted at 2026-08-10 18:52:25 on dist-test-slave-cbsf"
server_key: "e9a3ace730e0890075cdf40dcaa44723"
server_key_iv: "47fe5538cb1cafe03b631a8715d380df"
server_key_version: "encryptionkey@0"
I20260810 18:52:25.878777 8694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "dba3fe05861c4bbca11f744edd711f6a"
format_stamp: "Formatted at 2026-08-10 18:52:25 on dist-test-slave-cbsf"
server_key: "e9a3ace730e0890075cdf40dcaa44723"
server_key_iv: "47fe5538cb1cafe03b631a8715d380df"
server_key_version: "encryptionkey@0"
I20260810 18:52:25.882488 8694 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 18:52:25.884830 8711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:25.885943 8694 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:52:25.886092 8694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "dba3fe05861c4bbca11f744edd711f6a"
format_stamp: "Formatted at 2026-08-10 18:52:25 on dist-test-slave-cbsf"
server_key: "e9a3ace730e0890075cdf40dcaa44723"
server_key_iv: "47fe5538cb1cafe03b631a8715d380df"
server_key_version: "encryptionkey@0"
I20260810 18:52:25.886206 8694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:25.904376 8694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:25.907226 8694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:25.907450 8694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:25.914924 8694 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:35543
I20260810 18:52:25.914929 8765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:35543 every 8 connection(s)
I20260810 18:52:25.915855 8694 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:52:25.918861 8767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:25.921396 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 8694
I20260810 18:52:25.921574 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:52:25.921903 19106 external_mini_cluster.cc:1428] Setting key c38986cd1acaa32a5fe7de27e08e6d09
I20260810 18:52:25.924891 8767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dba3fe05861c4bbca11f744edd711f6a: Bootstrap starting.
I20260810 18:52:25.927007 8767 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dba3fe05861c4bbca11f744edd711f6a: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:25.927726 8767 log.cc:824] T 00000000000000000000000000000000 P dba3fe05861c4bbca11f744edd711f6a: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:25.929585 8767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dba3fe05861c4bbca11f744edd711f6a: No bootstrap required, opened a new log
Aug 10 18:52:25 dist-test-slave-cbsf krb5kdc[8679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:52:25.933676 8767 raft_consensus.cc:348] T 00000000000000000000000000000000 P dba3fe05861c4bbca11f744edd711f6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dba3fe05861c4bbca11f744edd711f6a" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 35543 } }
I20260810 18:52:25.933892 8767 raft_consensus.cc:374] T 00000000000000000000000000000000 P dba3fe05861c4bbca11f744edd711f6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:25.933925 8767 raft_consensus.cc:729] T 00000000000000000000000000000000 P dba3fe05861c4bbca11f744edd711f6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dba3fe05861c4bbca11f744edd711f6a, State: Initialized, Role: FOLLOWER
I20260810 18:52:25.934350 8767 consensus_queue.cc:260] T 00000000000000000000000000000000 P dba3fe05861c4bbca11f744edd711f6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dba3fe05861c4bbca11f744edd711f6a" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 35543 } }
I20260810 18:52:25.934468 8767 raft_consensus.cc:388] T 00000000000000000000000000000000 P dba3fe05861c4bbca11f744edd711f6a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:52:25.934547 8767 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba3fe05861c4bbca11f744edd711f6a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:52:25.934715 8767 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dba3fe05861c4bbca11f744edd711f6a [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:25.935891 8767 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba3fe05861c4bbca11f744edd711f6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dba3fe05861c4bbca11f744edd711f6a" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 35543 } }
I20260810 18:52:25.936266 8767 leader_election.cc:302] T 00000000000000000000000000000000 P dba3fe05861c4bbca11f744edd711f6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dba3fe05861c4bbca11f744edd711f6a; no voters:
I20260810 18:52:25.936676 8767 leader_election.cc:288] T 00000000000000000000000000000000 P dba3fe05861c4bbca11f744edd711f6a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:52:25.938172 8767 sys_catalog.cc:564] T 00000000000000000000000000000000 P dba3fe05861c4bbca11f744edd711f6a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:52:25.938257 8774 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dba3fe05861c4bbca11f744edd711f6a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:52:25.938438 8772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:25.923441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46546 (local address 127.18.168.190:35543)
0810 18:52:25.923931 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:25.923951 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:25.923987 (+ 36us) server_negotiation.cc:407] Connection header received
0810 18:52:25.924748 (+ 761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:25.924774 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:25.925186 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:25.925551 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:25.926450 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:25.927612 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:25.928636 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:25.929208 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:25.929398 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:25.932265 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:25.932271 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:25.932284 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:25.932313 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:25.934517 (+ 2204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:25.935063 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:25.935069 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:25.935075 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:25.935207 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:25.935561 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:25.935567 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:25.935572 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:25.936155 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:25.936377 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:25.936905 (+ 528us) server_negotiation.cc:299] Negotiation successful
0810 18:52:25.937423 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":164,"threads_started":1}
I20260810 18:52:25.938493 8774 raft_consensus.cc:686] T 00000000000000000000000000000000 P dba3fe05861c4bbca11f744edd711f6a [term 1 LEADER]: Becoming Leader. State: Replica: dba3fe05861c4bbca11f744edd711f6a, State: Running, Role: LEADER
I20260810 18:52:25.940090 8774 consensus_queue.cc:237] T 00000000000000000000000000000000 P dba3fe05861c4bbca11f744edd711f6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dba3fe05861c4bbca11f744edd711f6a" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 35543 } }
I20260810 18:52:25.941917 8776 sys_catalog.cc:455] T 00000000000000000000000000000000 P dba3fe05861c4bbca11f744edd711f6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dba3fe05861c4bbca11f744edd711f6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dba3fe05861c4bbca11f744edd711f6a" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 35543 } } }
I20260810 18:52:25.942065 8776 sys_catalog.cc:458] T 00000000000000000000000000000000 P dba3fe05861c4bbca11f744edd711f6a [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:25.942116 8775 sys_catalog.cc:455] T 00000000000000000000000000000000 P dba3fe05861c4bbca11f744edd711f6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader dba3fe05861c4bbca11f744edd711f6a. Latest consensus state: current_term: 1 leader_uuid: "dba3fe05861c4bbca11f744edd711f6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dba3fe05861c4bbca11f744edd711f6a" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 35543 } } }
I20260810 18:52:25.942235 8775 sys_catalog.cc:458] T 00000000000000000000000000000000 P dba3fe05861c4bbca11f744edd711f6a [sys.catalog]: This master's current role is: LEADER
W20260810 18:52:25.945408 8790 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dba3fe05861c4bbca11f744edd711f6a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:52:25.945501 8790 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:52:25.945577 8789 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:52:25.946301 8789 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:52:25.952126 8789 catalog_manager.cc:1347] Generated new cluster ID: e0bcce510d8941318a02ff4aad85d589
I20260810 18:52:25.952198 8789 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:52:25.982139 8789 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:52:25.983129 8789 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:52:25.992599 8789 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dba3fe05861c4bbca11f744edd711f6a: Generated new TSK 0
I20260810 18:52:25.993361 8789 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.168.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab.
Entry for principal kudu/127.18.168.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab.
I20260810 18:52:26.035655 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.129:0
--local_ip_for_outbound_sockets=127.18.168.129
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:35543
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:36543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:52:26.148586 8798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:26.148888 8798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:26.148936 8798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:26.152849 8798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:26.152899 8798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:26.152953 8798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.129
I20260810 18:52:26.158020 8798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:36543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.129.keytab
--principal=kudu/127.18.168.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:35543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.129
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:26.158950 8798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:26.159929 8798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:26.169497 8805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:26.169523 8808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:26.169538 8806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:26.169778 8798 server_base.cc:1034] running on GCE node
I20260810 18:52:26.170135 8798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:26.170778 8798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:26.171960 8798 hybrid_clock.cc:648] HybridClock initialized: now 1786387946171940 us; error 30 us; skew 500 ppm
Aug 10 18:52:26 dist-test-slave-cbsf krb5kdc[8679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387946, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:26.174690 8798 init.cc:377] Logged in from keytab as kudu/127.18.168.129@KRBTEST.COM (short username kudu)
I20260810 18:52:26.175539 8798 webserver.cc:460] Webserver started at http://127.18.168.129:33135/ using document root <none> and password file <none>
I20260810 18:52:26.176088 8798 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:26.176199 8798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:26.176437 8798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:26.178577 8798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-0/data/instance:
uuid: "79c24fe63325436b8dd0fb2b499e4853"
format_stamp: "Formatted at 2026-08-10 18:52:26 on dist-test-slave-cbsf"
server_key: "58da937ed2d000bd4113f1671cffaf00"
server_key_iv: "ceb7d06aee2ee9c2c119b44c0f9133f7"
server_key_version: "encryptionkey@0"
I20260810 18:52:26.179164 8798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance:
uuid: "79c24fe63325436b8dd0fb2b499e4853"
format_stamp: "Formatted at 2026-08-10 18:52:26 on dist-test-slave-cbsf"
server_key: "58da937ed2d000bd4113f1671cffaf00"
server_key_iv: "ceb7d06aee2ee9c2c119b44c0f9133f7"
server_key_version: "encryptionkey@0"
I20260810 18:52:26.182863 8798 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 18:52:26.185247 8815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:26.186487 8798 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:52:26.186633 8798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-0/wal
uuid: "79c24fe63325436b8dd0fb2b499e4853"
format_stamp: "Formatted at 2026-08-10 18:52:26 on dist-test-slave-cbsf"
server_key: "58da937ed2d000bd4113f1671cffaf00"
server_key_iv: "ceb7d06aee2ee9c2c119b44c0f9133f7"
server_key_version: "encryptionkey@0"
I20260810 18:52:26.186739 8798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:26.199182 8798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:26.202287 8798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:26.202508 8798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:26.203078 8798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:52:26.203924 8798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:52:26.203992 8798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:26.204058 8798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:52:26.204104 8798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:26.213182 8798 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.129:39527
I20260810 18:52:26.213291 8941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.129:39527 every 8 connection(s)
I20260810 18:52:26.214051 8798 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
I20260810 18:52:26.223017 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 8798
I20260810 18:52:26.223168 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance
Aug 10 18:52:26 dist-test-slave-cbsf krb5kdc[8679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387946, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.129@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:52:26.223492 19106 external_mini_cluster.cc:1428] Setting key 72f0b954f8fa2a976b39db4d36d5852a
I20260810 18:52:26.228040 8772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:26.215825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:33269 (local address 127.18.168.190:35543)
0810 18:52:26.215978 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:26.215983 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:26.217216 (+ 1233us) server_negotiation.cc:407] Connection header received
0810 18:52:26.218039 (+ 823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:26.218041 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:26.218088 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:26.218163 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:26.219400 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:26.219935 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:26.221098 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:26.221323 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:26.221403 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:26.223904 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:26.223908 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:26.223910 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:26.223932 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:26.225822 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:26.226307 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:26.226313 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:26.226318 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:26.226411 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:26.226744 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:26.226750 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:26.226754 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:26.227003 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:26.227107 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:26.227708 (+ 601us) server_negotiation.cc:299] Negotiation successful
0810 18:52:26.227840 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 18:52:26.228945 8946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:26.216176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:35543 (local address 127.18.168.129:33269)
0810 18:52:26.217030 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:26.217071 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:26.217839 (+ 768us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:26.218372 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:26.218382 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:26.218722 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:52:26.219271 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:26.219283 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:26.220039 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:26.220041 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:26.220990 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:26.221000 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:26.221452 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:26.221455 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:26.221586 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:26.222158 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:52:26.222173 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:52:26.223739 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:52:26.225952 (+ 2213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:26.225956 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:26.225967 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:26.226210 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:26.226507 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:26.226509 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:26.226510 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:26.226641 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:26.227140 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:52:26.227145 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:52:26.227435 (+ 290us) client_negotiation.cc:769] Sending connection context
0810 18:52:26.227651 (+ 216us) client_negotiation.cc:240] Negotiation successful
0810 18:52:26.227978 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":683,"thread_start_us":549,"threads_started":1}
I20260810 18:52:26.230176 8943 heartbeater.cc:347] Connected to a master server at 127.18.168.190:35543
I20260810 18:52:26.230424 8943 heartbeater.cc:464] Registering TS with master...
I20260810 18:52:26.231017 8943 heartbeater.cc:511] Master 127.18.168.190:35543 requested a full tablet report, sending...
I20260810 18:52:26.232802 8728 ts_manager.cc:194] Registered new tserver with Master: 79c24fe63325436b8dd0fb2b499e4853 (127.18.168.129:39527)
I20260810 18:52:26.234385 8728 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.129@KRBTEST.COM'} at 127.18.168.129:33269
WARNING: no policy specified for kudu/127.18.168.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab.
Entry for principal kudu/127.18.168.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab.
I20260810 18:52:26.253750 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.130:0
--local_ip_for_outbound_sockets=127.18.168.130
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:35543
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:36543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:52:26.367033 8949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:26.367278 8949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:26.367323 8949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:26.371402 8949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:26.371460 8949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:26.371517 8949 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.130
I20260810 18:52:26.376529 8949 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:36543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.130.keytab
--principal=kudu/127.18.168.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:35543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8949
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.130
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:26.377509 8949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:26.378443 8949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:26.387799 8954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:26.387841 8957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:26.387806 8955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:26.388118 8949 server_base.cc:1034] running on GCE node
I20260810 18:52:26.388595 8949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:26.389325 8949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:26.390511 8949 hybrid_clock.cc:648] HybridClock initialized: now 1786387946390464 us; error 58 us; skew 500 ppm
Aug 10 18:52:26 dist-test-slave-cbsf krb5kdc[8679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387946, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:26.393471 8949 init.cc:377] Logged in from keytab as kudu/127.18.168.130@KRBTEST.COM (short username kudu)
I20260810 18:52:26.394348 8949 webserver.cc:460] Webserver started at http://127.18.168.130:45813/ using document root <none> and password file <none>
I20260810 18:52:26.394913 8949 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:26.394979 8949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:26.395211 8949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:26.397363 8949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-1/data/instance:
uuid: "235e54b35f5e4534a0c2cd3c8107edd7"
format_stamp: "Formatted at 2026-08-10 18:52:26 on dist-test-slave-cbsf"
server_key: "a3eb124082867d75a121df7ba38d9477"
server_key_iv: "23eea9a14b10b00e398531ed20f3d16f"
server_key_version: "encryptionkey@0"
I20260810 18:52:26.397908 8949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance:
uuid: "235e54b35f5e4534a0c2cd3c8107edd7"
format_stamp: "Formatted at 2026-08-10 18:52:26 on dist-test-slave-cbsf"
server_key: "a3eb124082867d75a121df7ba38d9477"
server_key_iv: "23eea9a14b10b00e398531ed20f3d16f"
server_key_version: "encryptionkey@0"
I20260810 18:52:26.401604 8949 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 18:52:26.403882 8964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:26.405128 8949 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 18:52:26.405262 8949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-1/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-1/wal
uuid: "235e54b35f5e4534a0c2cd3c8107edd7"
format_stamp: "Formatted at 2026-08-10 18:52:26 on dist-test-slave-cbsf"
server_key: "a3eb124082867d75a121df7ba38d9477"
server_key_iv: "23eea9a14b10b00e398531ed20f3d16f"
server_key_version: "encryptionkey@0"
I20260810 18:52:26.405367 8949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:26.435133 8949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:26.438278 8949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:26.438576 8949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:26.439211 8949 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:52:26.440069 8949 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:52:26.440140 8949 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:26.440253 8949 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:52:26.440284 8949 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:26.449584 8949 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.130:35895
I20260810 18:52:26.449622 9094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.130:35895 every 8 connection(s)
I20260810 18:52:26.450417 8949 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
I20260810 18:52:26.459906 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 8949
I20260810 18:52:26.460067 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance
Aug 10 18:52:26 dist-test-slave-cbsf krb5kdc[8679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387946, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.130@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:52:26.460399 19106 external_mini_cluster.cc:1428] Setting key 89c1386aa8ac575f8b0bf55189a7be5d
I20260810 18:52:26.464546 8772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:26.452158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:50693 (local address 127.18.168.190:35543)
0810 18:52:26.452286 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:26.452289 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:26.453045 (+ 756us) server_negotiation.cc:407] Connection header received
0810 18:52:26.454359 (+ 1314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:26.454362 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:26.454421 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:26.454524 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:26.455627 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:26.456402 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:26.457789 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:26.458027 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:26.458144 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:26.461022 (+ 2878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:26.461026 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:26.461029 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:26.461059 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:26.462541 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:26.463036 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:26.463039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:26.463041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:26.463124 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:26.463463 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:26.463466 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:26.463467 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:26.463644 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:26.463768 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:26.464246 (+ 478us) server_negotiation.cc:299] Negotiation successful
0810 18:52:26.464366 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 18:52:26.465480 9097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:26.452448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:35543 (local address 127.18.168.130:50693)
0810 18:52:26.452888 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:26.452928 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:26.454150 (+ 1222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:26.454692 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:26.454698 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:26.455039 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:52:26.455479 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:26.455491 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:26.456609 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:26.456614 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:26.457641 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:26.457651 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:26.458200 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:26.458204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:26.458379 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:26.458995 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:52:26.459012 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:52:26.460841 (+ 1829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:52:26.462663 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:26.462668 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:26.462681 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:26.462931 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:26.463224 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:26.463227 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:26.463229 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:26.463364 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:26.463768 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:52:26.463775 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:52:26.464014 (+ 239us) client_negotiation.cc:769] Sending connection context
0810 18:52:26.464188 (+ 174us) client_negotiation.cc:240] Negotiation successful
0810 18:52:26.464491 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":103,"threads_started":1}
I20260810 18:52:26.466709 9095 heartbeater.cc:347] Connected to a master server at 127.18.168.190:35543
I20260810 18:52:26.466943 9095 heartbeater.cc:464] Registering TS with master...
I20260810 18:52:26.467408 9095 heartbeater.cc:511] Master 127.18.168.190:35543 requested a full tablet report, sending...
I20260810 18:52:26.468482 8728 ts_manager.cc:194] Registered new tserver with Master: 235e54b35f5e4534a0c2cd3c8107edd7 (127.18.168.130:35895)
I20260810 18:52:26.469484 8728 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.130@KRBTEST.COM'} at 127.18.168.130:50693
WARNING: no policy specified for kudu/127.18.168.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab.
Entry for principal kudu/127.18.168.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab.
I20260810 18:52:26.491638 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.131:0
--local_ip_for_outbound_sockets=127.18.168.131
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:35543
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:36543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:52:26.603483 9101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:26.603704 9101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:26.603740 9101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:26.607666 9101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:26.607714 9101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:26.607766 9101 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.131
I20260810 18:52:26.612887 9101 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:36543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.131.keytab
--principal=kudu/127.18.168.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:35543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.131
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:26.613735 9101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:26.614615 9101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:26.624048 9109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:26.624061 9106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:26.624136 9101 server_base.cc:1034] running on GCE node
W20260810 18:52:26.624058 9107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:26.624761 9101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:26.625396 9101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:26.626619 9101 hybrid_clock.cc:648] HybridClock initialized: now 1786387946626587 us; error 40 us; skew 500 ppm
Aug 10 18:52:26 dist-test-slave-cbsf krb5kdc[8679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387946, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:26.629367 9101 init.cc:377] Logged in from keytab as kudu/127.18.168.131@KRBTEST.COM (short username kudu)
I20260810 18:52:26.630182 9101 webserver.cc:460] Webserver started at http://127.18.168.131:43405/ using document root <none> and password file <none>
I20260810 18:52:26.630715 9101 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:26.630827 9101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:26.631081 9101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:26.633165 9101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-2/data/instance:
uuid: "f01b020d35db4744bd395053de3200c8"
format_stamp: "Formatted at 2026-08-10 18:52:26 on dist-test-slave-cbsf"
server_key: "a4077e75093413b1a8e508b1cced32cc"
server_key_iv: "a23a19d1be563941875f447665c426a5"
server_key_version: "encryptionkey@0"
I20260810 18:52:26.633738 9101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance:
uuid: "f01b020d35db4744bd395053de3200c8"
format_stamp: "Formatted at 2026-08-10 18:52:26 on dist-test-slave-cbsf"
server_key: "a4077e75093413b1a8e508b1cced32cc"
server_key_iv: "a23a19d1be563941875f447665c426a5"
server_key_version: "encryptionkey@0"
I20260810 18:52:26.637513 9101 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:52:26.639964 9116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:26.641247 9101 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:52:26.641383 9101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-2/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-2/wal
uuid: "f01b020d35db4744bd395053de3200c8"
format_stamp: "Formatted at 2026-08-10 18:52:26 on dist-test-slave-cbsf"
server_key: "a4077e75093413b1a8e508b1cced32cc"
server_key_iv: "a23a19d1be563941875f447665c426a5"
server_key_version: "encryptionkey@0"
I20260810 18:52:26.641492 9101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:26.654251 9101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:26.657030 9101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:26.657248 9101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:26.657783 9101 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:52:26.658579 9101 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:52:26.658648 9101 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:26.658715 9101 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:52:26.658761 9101 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:26.667989 9101 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.131:44709
I20260810 18:52:26.668012 9243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.131:44709 every 8 connection(s)
I20260810 18:52:26.668907 9101 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
Aug 10 18:52:26 dist-test-slave-cbsf krb5kdc[8679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387946, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.131@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:52:26.678288 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 9101
I20260810 18:52:26.678470 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance
I20260810 18:52:26.678782 19106 external_mini_cluster.cc:1428] Setting key 8e2d545f231e399b82cf229be6c718e6
I20260810 18:52:26.683223 8772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:26.670690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:50735 (local address 127.18.168.190:35543)
0810 18:52:26.670829 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:26.670833 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:26.671625 (+ 792us) server_negotiation.cc:407] Connection header received
0810 18:52:26.673003 (+ 1378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:26.673007 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:26.673067 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:26.673179 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:26.674291 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:26.674827 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:26.675996 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:26.676237 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:26.676347 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:26.679084 (+ 2737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:26.679088 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:26.679091 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:26.679134 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:26.680946 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:26.681529 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:26.681539 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:26.681540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:26.681629 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:26.682015 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:26.682019 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:26.682021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:26.682231 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:26.682368 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:26.682891 (+ 523us) server_negotiation.cc:299] Negotiation successful
0810 18:52:26.683034 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 18:52:26.683956 9247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:26.671038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:35543 (local address 127.18.168.131:50735)
0810 18:52:26.671463 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:26.671504 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:26.672786 (+ 1282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:26.673355 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:26.673362 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:26.673695 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:52:26.674144 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:26.674158 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:26.674985 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:26.674993 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:26.675865 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:26.675875 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:26.676397 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:26.676401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:26.676559 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:26.677208 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:52:26.677225 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:52:26.678857 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:52:26.681088 (+ 2231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:26.681094 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:26.681109 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:26.681392 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:26.681742 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:26.681746 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:26.681748 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:26.681898 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:26.682367 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:52:26.682374 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:52:26.682630 (+ 256us) client_negotiation.cc:769] Sending connection context
0810 18:52:26.682829 (+ 199us) client_negotiation.cc:240] Negotiation successful
0810 18:52:26.683155 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":109,"threads_started":1}
I20260810 18:52:26.685321 9245 heartbeater.cc:347] Connected to a master server at 127.18.168.190:35543
I20260810 18:52:26.685598 9245 heartbeater.cc:464] Registering TS with master...
I20260810 18:52:26.686110 9245 heartbeater.cc:511] Master 127.18.168.190:35543 requested a full tablet report, sending...
I20260810 18:52:26.687147 8728 ts_manager.cc:194] Registered new tserver with Master: f01b020d35db4744bd395053de3200c8 (127.18.168.131:44709)
I20260810 18:52:26.687713 8728 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.168.131@KRBTEST.COM'} at 127.18.168.131:50735
I20260810 18:52:26.692883 19106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:52:26.701552 8772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:26.694821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46562 (local address 127.18.168.190:35543)
0810 18:52:26.694959 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:26.694963 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:26.695092 (+ 129us) server_negotiation.cc:407] Connection header received
0810 18:52:26.695181 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:26.695183 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:26.695217 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:26.695299 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:26.695916 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:26.696384 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:26.697253 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:26.697456 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:26.697531 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:26.698693 (+ 1162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:26.698695 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:26.698697 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:26.698717 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:26.700130 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:26.700461 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:26.700463 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:26.700464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:26.700531 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:26.700798 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:26.700801 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:26.700803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:26.701031 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:26.701158 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:26.701355 (+ 197us) server_negotiation.cc:299] Negotiation successful
0810 18:52:26.701448 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 18:52:26.704144 8728 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46562:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:52:26.707105 8728 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:52:26.721750 9257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:26.715639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.131:44709 (local address 127.0.0.1:58312)
0810 18:52:26.716359 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:26.716374 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:26.716470 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:26.717334 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:26.717340 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:26.717364 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:26.717495 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:26.717501 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:26.718705 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:26.718708 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:26.720013 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:26.720037 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:26.721194 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:26.721198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:26.721311 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:26.721357 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 18:52:26.721460 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 18:52:26.721592 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"mutex_wait_us":12,"thread_start_us":93,"threads_started":1}
I20260810 18:52:26.721918 9259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:26.716365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:35895 (local address 127.0.0.1:58098)
0810 18:52:26.717266 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:26.717276 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:26.717343 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:26.717850 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:26.717854 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:26.717873 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:26.717976 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:26.717982 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:26.719417 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:26.719420 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:26.720560 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:26.720569 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:26.721487 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:26.721493 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:26.721615 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:26.721646 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 18:52:26.721714 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 18:52:26.721782 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":846,"mutex_wait_us":24,"thread_start_us":111,"threads_started":1}
I20260810 18:52:26.721941 9260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:26.716459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58312 (local address 127.18.168.131:44709)
0810 18:52:26.716845 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:26.716863 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:26.716881 (+ 18us) server_negotiation.cc:407] Connection header received
0810 18:52:26.716972 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:26.716978 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:26.717161 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:26.717341 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:26.717608 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:26.718564 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:26.720149 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:26.721066 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:26.721186 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:26.721303 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:26.721693 (+ 390us) server_negotiation.cc:299] Negotiation successful
0810 18:52:26.721796 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":180,"threads_started":1}
I20260810 18:52:26.722374 9261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:26.716809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58098 (local address 127.18.168.130:35895)
0810 18:52:26.717526 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:26.717534 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:26.717550 (+ 16us) server_negotiation.cc:407] Connection header received
0810 18:52:26.717606 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:26.717611 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:26.717732 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:26.717835 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:26.718094 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:26.719257 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:26.720675 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:26.721356 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:26.721487 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:26.721603 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:26.722142 (+ 539us) server_negotiation.cc:299] Negotiation successful
0810 18:52:26.722240 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":129,"threads_started":1}
I20260810 18:52:26.723331 9256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:26.715626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:39527 (local address 127.0.0.1:51944)
0810 18:52:26.715978 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:26.716029 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:26.716161 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:26.718592 (+ 2431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:26.718596 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:26.718618 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:26.718718 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:26.718724 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:26.720284 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:26.720287 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:26.721402 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:26.721433 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:26.722960 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:26.722963 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:26.723052 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:26.723077 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 18:52:26.723154 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 18:52:26.723216 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":96,"threads_started":1}
I20260810 18:52:26.723424 9258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:26.715780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51944 (local address 127.18.168.129:39527)
0810 18:52:26.718155 (+ 2375us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:26.718170 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:26.718193 (+ 23us) server_negotiation.cc:407] Connection header received
0810 18:52:26.718267 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:26.718274 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:26.718451 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:26.718589 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:26.718831 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:26.720149 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:26.721561 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:26.722622 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:26.722754 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:26.722879 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:26.723203 (+ 324us) server_negotiation.cc:299] Negotiation successful
0810 18:52:26.723295 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2234,"thread_start_us":91,"threads_started":1}
I20260810 18:52:26.724999 9171 tablet_service.cc:1467] Processing CreateTablet for tablet c3ced64f204d4a62b80e705a9c234136 (DEFAULT_TABLE table=test-table [id=99fc34ed1ca54548b5ab001e687ad682]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:52:26.725147 9019 tablet_service.cc:1467] Processing CreateTablet for tablet c3ced64f204d4a62b80e705a9c234136 (DEFAULT_TABLE table=test-table [id=99fc34ed1ca54548b5ab001e687ad682]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:52:26.726049 9171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3ced64f204d4a62b80e705a9c234136. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:26.726190 9019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3ced64f204d4a62b80e705a9c234136. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:26.727329 8866 tablet_service.cc:1467] Processing CreateTablet for tablet c3ced64f204d4a62b80e705a9c234136 (DEFAULT_TABLE table=test-table [id=99fc34ed1ca54548b5ab001e687ad682]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:52:26.728210 8866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3ced64f204d4a62b80e705a9c234136. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:26.732187 9262 tablet_bootstrap.cc:492] T c3ced64f204d4a62b80e705a9c234136 P f01b020d35db4744bd395053de3200c8: Bootstrap starting.
I20260810 18:52:26.734313 9262 tablet_bootstrap.cc:654] T c3ced64f204d4a62b80e705a9c234136 P f01b020d35db4744bd395053de3200c8: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:26.734653 9264 tablet_bootstrap.cc:492] T c3ced64f204d4a62b80e705a9c234136 P 79c24fe63325436b8dd0fb2b499e4853: Bootstrap starting.
I20260810 18:52:26.735260 9263 tablet_bootstrap.cc:492] T c3ced64f204d4a62b80e705a9c234136 P 235e54b35f5e4534a0c2cd3c8107edd7: Bootstrap starting.
I20260810 18:52:26.735661 9262 log.cc:824] T c3ced64f204d4a62b80e705a9c234136 P f01b020d35db4744bd395053de3200c8: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:26.736843 9264 tablet_bootstrap.cc:654] T c3ced64f204d4a62b80e705a9c234136 P 79c24fe63325436b8dd0fb2b499e4853: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:26.737372 9263 tablet_bootstrap.cc:654] T c3ced64f204d4a62b80e705a9c234136 P 235e54b35f5e4534a0c2cd3c8107edd7: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:26.737774 9264 log.cc:824] T c3ced64f204d4a62b80e705a9c234136 P 79c24fe63325436b8dd0fb2b499e4853: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:26.738075 9262 tablet_bootstrap.cc:492] T c3ced64f204d4a62b80e705a9c234136 P f01b020d35db4744bd395053de3200c8: No bootstrap required, opened a new log
I20260810 18:52:26.738282 9262 ts_tablet_manager.cc:1397] T c3ced64f204d4a62b80e705a9c234136 P f01b020d35db4744bd395053de3200c8: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260810 18:52:26.738765 9263 log.cc:824] T c3ced64f204d4a62b80e705a9c234136 P 235e54b35f5e4534a0c2cd3c8107edd7: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:26.739859 9264 tablet_bootstrap.cc:492] T c3ced64f204d4a62b80e705a9c234136 P 79c24fe63325436b8dd0fb2b499e4853: No bootstrap required, opened a new log
I20260810 18:52:26.740094 9264 ts_tablet_manager.cc:1397] T c3ced64f204d4a62b80e705a9c234136 P 79c24fe63325436b8dd0fb2b499e4853: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260810 18:52:26.740713 9263 tablet_bootstrap.cc:492] T c3ced64f204d4a62b80e705a9c234136 P 235e54b35f5e4534a0c2cd3c8107edd7: No bootstrap required, opened a new log
I20260810 18:52:26.740888 9263 ts_tablet_manager.cc:1397] T c3ced64f204d4a62b80e705a9c234136 P 235e54b35f5e4534a0c2cd3c8107edd7: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260810 18:52:26.742247 9262 raft_consensus.cc:348] T c3ced64f204d4a62b80e705a9c234136 P f01b020d35db4744bd395053de3200c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c24fe63325436b8dd0fb2b499e4853" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39527 } } peers { permanent_uuid: "f01b020d35db4744bd395053de3200c8" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 44709 } } peers { permanent_uuid: "235e54b35f5e4534a0c2cd3c8107edd7" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35895 } }
I20260810 18:52:26.742487 9262 raft_consensus.cc:374] T c3ced64f204d4a62b80e705a9c234136 P f01b020d35db4744bd395053de3200c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:26.742533 9262 raft_consensus.cc:729] T c3ced64f204d4a62b80e705a9c234136 P f01b020d35db4744bd395053de3200c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f01b020d35db4744bd395053de3200c8, State: Initialized, Role: FOLLOWER
I20260810 18:52:26.742959 9262 consensus_queue.cc:260] T c3ced64f204d4a62b80e705a9c234136 P f01b020d35db4744bd395053de3200c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c24fe63325436b8dd0fb2b499e4853" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39527 } } peers { permanent_uuid: "f01b020d35db4744bd395053de3200c8" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 44709 } } peers { permanent_uuid: "235e54b35f5e4534a0c2cd3c8107edd7" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35895 } }
I20260810 18:52:26.743397 9264 raft_consensus.cc:348] T c3ced64f204d4a62b80e705a9c234136 P 79c24fe63325436b8dd0fb2b499e4853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c24fe63325436b8dd0fb2b499e4853" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39527 } } peers { permanent_uuid: "f01b020d35db4744bd395053de3200c8" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 44709 } } peers { permanent_uuid: "235e54b35f5e4534a0c2cd3c8107edd7" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35895 } }
I20260810 18:52:26.743619 9245 heartbeater.cc:503] Master 127.18.168.190:35543 was elected leader, sending a full tablet report...
I20260810 18:52:26.743644 9264 raft_consensus.cc:374] T c3ced64f204d4a62b80e705a9c234136 P 79c24fe63325436b8dd0fb2b499e4853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:26.743690 9264 raft_consensus.cc:729] T c3ced64f204d4a62b80e705a9c234136 P 79c24fe63325436b8dd0fb2b499e4853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79c24fe63325436b8dd0fb2b499e4853, State: Initialized, Role: FOLLOWER
I20260810 18:52:26.744097 9264 consensus_queue.cc:260] T c3ced64f204d4a62b80e705a9c234136 P 79c24fe63325436b8dd0fb2b499e4853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c24fe63325436b8dd0fb2b499e4853" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39527 } } peers { permanent_uuid: "f01b020d35db4744bd395053de3200c8" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 44709 } } peers { permanent_uuid: "235e54b35f5e4534a0c2cd3c8107edd7" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35895 } }
I20260810 18:52:26.744684 9262 ts_tablet_manager.cc:1428] T c3ced64f204d4a62b80e705a9c234136 P f01b020d35db4744bd395053de3200c8: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 18:52:26.744489 9263 raft_consensus.cc:348] T c3ced64f204d4a62b80e705a9c234136 P 235e54b35f5e4534a0c2cd3c8107edd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c24fe63325436b8dd0fb2b499e4853" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39527 } } peers { permanent_uuid: "f01b020d35db4744bd395053de3200c8" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 44709 } } peers { permanent_uuid: "235e54b35f5e4534a0c2cd3c8107edd7" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35895 } }
I20260810 18:52:26.744871 9263 raft_consensus.cc:374] T c3ced64f204d4a62b80e705a9c234136 P 235e54b35f5e4534a0c2cd3c8107edd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:26.744993 9263 raft_consensus.cc:729] T c3ced64f204d4a62b80e705a9c234136 P 235e54b35f5e4534a0c2cd3c8107edd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235e54b35f5e4534a0c2cd3c8107edd7, State: Initialized, Role: FOLLOWER
I20260810 18:52:26.745122 9264 ts_tablet_manager.cc:1428] T c3ced64f204d4a62b80e705a9c234136 P 79c24fe63325436b8dd0fb2b499e4853: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260810 18:52:26.745290 8943 heartbeater.cc:503] Master 127.18.168.190:35543 was elected leader, sending a full tablet report...
I20260810 18:52:26.745409 9263 consensus_queue.cc:260] T c3ced64f204d4a62b80e705a9c234136 P 235e54b35f5e4534a0c2cd3c8107edd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c24fe63325436b8dd0fb2b499e4853" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39527 } } peers { permanent_uuid: "f01b020d35db4744bd395053de3200c8" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 44709 } } peers { permanent_uuid: "235e54b35f5e4534a0c2cd3c8107edd7" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35895 } }
I20260810 18:52:26.746074 9095 heartbeater.cc:503] Master 127.18.168.190:35543 was elected leader, sending a full tablet report...
I20260810 18:52:26.746660 9263 ts_tablet_manager.cc:1428] T c3ced64f204d4a62b80e705a9c234136 P 235e54b35f5e4534a0c2cd3c8107edd7: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260810 18:52:26.813081 9270 raft_consensus.cc:482] T c3ced64f204d4a62b80e705a9c234136 P 79c24fe63325436b8dd0fb2b499e4853 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:52:26.813311 9270 raft_consensus.cc:504] T c3ced64f204d4a62b80e705a9c234136 P 79c24fe63325436b8dd0fb2b499e4853 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c24fe63325436b8dd0fb2b499e4853" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39527 } } peers { permanent_uuid: "f01b020d35db4744bd395053de3200c8" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 44709 } } peers { permanent_uuid: "235e54b35f5e4534a0c2cd3c8107edd7" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35895 } }
I20260810 18:52:26.814538 9270 leader_election.cc:288] T c3ced64f204d4a62b80e705a9c234136 P 79c24fe63325436b8dd0fb2b499e4853 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f01b020d35db4744bd395053de3200c8 (127.18.168.131:44709), 235e54b35f5e4534a0c2cd3c8107edd7 (127.18.168.130:35895)
I20260810 18:52:26.819908 9260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:26.814917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:48699 (local address 127.18.168.131:44709)
0810 18:52:26.815053 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:26.815058 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:26.815198 (+ 140us) server_negotiation.cc:407] Connection header received
0810 18:52:26.815345 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:26.815351 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:26.815408 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:26.815529 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:26.815895 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:26.816744 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:26.818571 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:26.819182 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:26.819316 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:26.819360 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:26.819676 (+ 316us) server_negotiation.cc:299] Negotiation successful
0810 18:52:26.819771 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 18:52:26.819921 9272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:26.814747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.131:44709 (local address 127.18.168.129:48699)
0810 18:52:26.815078 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:26.815093 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:26.815224 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:26.815541 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:26.815544 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:26.815560 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:26.815713 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:26.815721 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:26.816863 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:26.816866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:26.818443 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:26.818454 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:26.819306 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:26.819309 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:26.819487 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:26.819560 (+ 73us) client_negotiation.cc:769] Sending connection context
0810 18:52:26.819645 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 18:52:26.819731 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":193,"thread_start_us":131,"threads_started":1}
I20260810 18:52:26.819975 9261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:26.815145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:49407 (local address 127.18.168.130:35895)
0810 18:52:26.815279 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:26.815285 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:26.815303 (+ 18us) server_negotiation.cc:407] Connection header received
0810 18:52:26.815493 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:26.815496 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:26.815555 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:26.815630 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:26.815919 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:26.816656 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:26.818571 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:26.819182 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:26.819306 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:26.819360 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:26.819747 (+ 387us) server_negotiation.cc:299] Negotiation successful
0810 18:52:26.819855 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 18:52:26.820276 9273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:26.814769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:35895 (local address 127.18.168.129:49407)
0810 18:52:26.815178 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:26.815195 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:26.815317 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:26.815683 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:26.815686 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:26.815699 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:26.815822 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:26.815831 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:26.816823 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:26.816829 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:26.818443 (+ 1614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:26.818454 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:26.819306 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:26.819309 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:26.819483 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:26.819560 (+ 77us) client_negotiation.cc:769] Sending connection context
0810 18:52:26.820103 (+ 543us) client_negotiation.cc:240] Negotiation successful
0810 18:52:26.820186 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"mutex_wait_us":178,"thread_start_us":128,"threads_started":1}
I20260810 18:52:26.820761 9192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3ced64f204d4a62b80e705a9c234136" candidate_uuid: "79c24fe63325436b8dd0fb2b499e4853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f01b020d35db4744bd395053de3200c8" is_pre_election: true
I20260810 18:52:26.821143 9192 raft_consensus.cc:2454] T c3ced64f204d4a62b80e705a9c234136 P f01b020d35db4744bd395053de3200c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79c24fe63325436b8dd0fb2b499e4853 in term 0.
I20260810 18:52:26.821183 9041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3ced64f204d4a62b80e705a9c234136" candidate_uuid: "79c24fe63325436b8dd0fb2b499e4853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235e54b35f5e4534a0c2cd3c8107edd7" is_pre_election: true
I20260810 18:52:26.821506 9041 raft_consensus.cc:2454] T c3ced64f204d4a62b80e705a9c234136 P 235e54b35f5e4534a0c2cd3c8107edd7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79c24fe63325436b8dd0fb2b499e4853 in term 0.
I20260810 18:52:26.821645 8818 leader_election.cc:302] T c3ced64f204d4a62b80e705a9c234136 P 79c24fe63325436b8dd0fb2b499e4853 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79c24fe63325436b8dd0fb2b499e4853, f01b020d35db4744bd395053de3200c8; no voters:
I20260810 18:52:26.821943 9270 raft_consensus.cc:2781] T c3ced64f204d4a62b80e705a9c234136 P 79c24fe63325436b8dd0fb2b499e4853 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:52:26.822050 9270 raft_consensus.cc:482] T c3ced64f204d4a62b80e705a9c234136 P 79c24fe63325436b8dd0fb2b499e4853 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:52:26.822094 9270 raft_consensus.cc:3037] T c3ced64f204d4a62b80e705a9c234136 P 79c24fe63325436b8dd0fb2b499e4853 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:26.823164 9270 raft_consensus.cc:504] T c3ced64f204d4a62b80e705a9c234136 P 79c24fe63325436b8dd0fb2b499e4853 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c24fe63325436b8dd0fb2b499e4853" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39527 } } peers { permanent_uuid: "f01b020d35db4744bd395053de3200c8" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 44709 } } peers { permanent_uuid: "235e54b35f5e4534a0c2cd3c8107edd7" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35895 } }
I20260810 18:52:26.823475 9270 leader_election.cc:288] T c3ced64f204d4a62b80e705a9c234136 P 79c24fe63325436b8dd0fb2b499e4853 [CANDIDATE]: Term 1 election: Requested vote from peers f01b020d35db4744bd395053de3200c8 (127.18.168.131:44709), 235e54b35f5e4534a0c2cd3c8107edd7 (127.18.168.130:35895)
I20260810 18:52:26.823794 9192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3ced64f204d4a62b80e705a9c234136" candidate_uuid: "79c24fe63325436b8dd0fb2b499e4853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f01b020d35db4744bd395053de3200c8"
I20260810 18:52:26.823868 9041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3ced64f204d4a62b80e705a9c234136" candidate_uuid: "79c24fe63325436b8dd0fb2b499e4853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235e54b35f5e4534a0c2cd3c8107edd7"
I20260810 18:52:26.823959 9192 raft_consensus.cc:3037] T c3ced64f204d4a62b80e705a9c234136 P f01b020d35db4744bd395053de3200c8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:26.824002 9041 raft_consensus.cc:3037] T c3ced64f204d4a62b80e705a9c234136 P 235e54b35f5e4534a0c2cd3c8107edd7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:26.825124 9192 raft_consensus.cc:2454] T c3ced64f204d4a62b80e705a9c234136 P f01b020d35db4744bd395053de3200c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79c24fe63325436b8dd0fb2b499e4853 in term 1.
I20260810 18:52:26.825243 9041 raft_consensus.cc:2454] T c3ced64f204d4a62b80e705a9c234136 P 235e54b35f5e4534a0c2cd3c8107edd7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79c24fe63325436b8dd0fb2b499e4853 in term 1.
I20260810 18:52:26.825459 8818 leader_election.cc:302] T c3ced64f204d4a62b80e705a9c234136 P 79c24fe63325436b8dd0fb2b499e4853 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79c24fe63325436b8dd0fb2b499e4853, f01b020d35db4744bd395053de3200c8; no voters:
I20260810 18:52:26.825696 9270 raft_consensus.cc:2781] T c3ced64f204d4a62b80e705a9c234136 P 79c24fe63325436b8dd0fb2b499e4853 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:52:26.825933 9270 raft_consensus.cc:686] T c3ced64f204d4a62b80e705a9c234136 P 79c24fe63325436b8dd0fb2b499e4853 [term 1 LEADER]: Becoming Leader. State: Replica: 79c24fe63325436b8dd0fb2b499e4853, State: Running, Role: LEADER
I20260810 18:52:26.826249 9270 consensus_queue.cc:237] T c3ced64f204d4a62b80e705a9c234136 P 79c24fe63325436b8dd0fb2b499e4853 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c24fe63325436b8dd0fb2b499e4853" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39527 } } peers { permanent_uuid: "f01b020d35db4744bd395053de3200c8" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 44709 } } peers { permanent_uuid: "235e54b35f5e4534a0c2cd3c8107edd7" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35895 } }
I20260810 18:52:26.829413 8728 catalog_manager.cc:5495] T c3ced64f204d4a62b80e705a9c234136 P 79c24fe63325436b8dd0fb2b499e4853 reported cstate change: term changed from 0 to 1, leader changed from <none> to 79c24fe63325436b8dd0fb2b499e4853 (127.18.168.129). New cstate: current_term: 1 leader_uuid: "79c24fe63325436b8dd0fb2b499e4853" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c24fe63325436b8dd0fb2b499e4853" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 39527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f01b020d35db4744bd395053de3200c8" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 44709 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "235e54b35f5e4534a0c2cd3c8107edd7" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35895 } health_report { overall_health: UNKNOWN } } }
I20260810 18:52:26.840972 9258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:26.837356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51960 (local address 127.18.168.129:39527)
0810 18:52:26.837468 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:26.837472 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:26.837484 (+ 12us) server_negotiation.cc:407] Connection header received
0810 18:52:26.837633 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:26.837635 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:26.837687 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:26.837770 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:52:26.838002 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:26.838561 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:26.839488 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:26.839691 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:26.839767 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:26.840089 (+ 322us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:52:26.840494 (+ 405us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:52:26.840602 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:26.840829 (+ 227us) server_negotiation.cc:299] Negotiation successful
0810 18:52:26.840877 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 18:52:26.846760 9041 raft_consensus.cc:1261] T c3ced64f204d4a62b80e705a9c234136 P 235e54b35f5e4534a0c2cd3c8107edd7 [term 1 FOLLOWER]: Refusing update from remote peer 79c24fe63325436b8dd0fb2b499e4853: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:52:26.846760 9192 raft_consensus.cc:1261] T c3ced64f204d4a62b80e705a9c234136 P f01b020d35db4744bd395053de3200c8 [term 1 FOLLOWER]: Refusing update from remote peer 79c24fe63325436b8dd0fb2b499e4853: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:52:26.847460 9270 consensus_queue.cc:1035] T c3ced64f204d4a62b80e705a9c234136 P 79c24fe63325436b8dd0fb2b499e4853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f01b020d35db4744bd395053de3200c8" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 44709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:52:26.847663 9274 consensus_queue.cc:1035] T c3ced64f204d4a62b80e705a9c234136 P 79c24fe63325436b8dd0fb2b499e4853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "235e54b35f5e4534a0c2cd3c8107edd7" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 35895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:52:26.865228 8727 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:52:26.865417 8727 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:52:26.867908 8727 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dba3fe05861c4bbca11f744edd711f6a: Sending DeleteTablet for 3 replicas of tablet c3ced64f204d4a62b80e705a9c234136
I20260810 18:52:26.868729 9171 tablet_service.cc:1514] Processing DeleteTablet for tablet c3ced64f204d4a62b80e705a9c234136 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:52:26 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:58312
I20260810 18:52:26.868775 9019 tablet_service.cc:1514] Processing DeleteTablet for tablet c3ced64f204d4a62b80e705a9c234136 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:52:26 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:58098
I20260810 18:52:26.868805 8866 tablet_service.cc:1514] Processing DeleteTablet for tablet c3ced64f204d4a62b80e705a9c234136 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:52:26 UTC) from {username='kudu', principal='kudu/127.18.168.190@KRBTEST.COM'} at 127.0.0.1:51944
I20260810 18:52:26.869267 9289 tablet_replica.cc:330] T c3ced64f204d4a62b80e705a9c234136 P 235e54b35f5e4534a0c2cd3c8107edd7: stopping tablet replica
I20260810 18:52:26.869274 9291 tablet_replica.cc:330] T c3ced64f204d4a62b80e705a9c234136 P 79c24fe63325436b8dd0fb2b499e4853: stopping tablet replica
I20260810 18:52:26.869575 9291 raft_consensus.cc:2229] T c3ced64f204d4a62b80e705a9c234136 P 79c24fe63325436b8dd0fb2b499e4853 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:52:26.869575 9289 raft_consensus.cc:2229] T c3ced64f204d4a62b80e705a9c234136 P 235e54b35f5e4534a0c2cd3c8107edd7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:52:26.869886 9289 raft_consensus.cc:2258] T c3ced64f204d4a62b80e705a9c234136 P 235e54b35f5e4534a0c2cd3c8107edd7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:52:26.869994 9291 raft_consensus.cc:2258] T c3ced64f204d4a62b80e705a9c234136 P 79c24fe63325436b8dd0fb2b499e4853 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:52:26.870082 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 8798
I20260810 18:52:26.873064 9289 ts_tablet_manager.cc:1905] T c3ced64f204d4a62b80e705a9c234136 P 235e54b35f5e4534a0c2cd3c8107edd7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:52:26.876574 9289 ts_tablet_manager.cc:1918] T c3ced64f204d4a62b80e705a9c234136 P 235e54b35f5e4534a0c2cd3c8107edd7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:52:26.876721 9289 log.cc:1196] T c3ced64f204d4a62b80e705a9c234136 P 235e54b35f5e4534a0c2cd3c8107edd7: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-1/wal/wals/c3ced64f204d4a62b80e705a9c234136
I20260810 18:52:26.876725 9290 tablet_replica.cc:330] T c3ced64f204d4a62b80e705a9c234136 P f01b020d35db4744bd395053de3200c8: stopping tablet replica
I20260810 18:52:26.877007 9290 raft_consensus.cc:2229] T c3ced64f204d4a62b80e705a9c234136 P f01b020d35db4744bd395053de3200c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:52:26.877162 9289 ts_tablet_manager.cc:1939] T c3ced64f204d4a62b80e705a9c234136 P 235e54b35f5e4534a0c2cd3c8107edd7: Deleting consensus metadata
I20260810 18:52:26.877380 9290 raft_consensus.cc:2258] T c3ced64f204d4a62b80e705a9c234136 P f01b020d35db4744bd395053de3200c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:52:26.878458 8713 catalog_manager.cc:4841] TS 235e54b35f5e4534a0c2cd3c8107edd7 (127.18.168.130:35895): tablet c3ced64f204d4a62b80e705a9c234136 (table test-table [id=99fc34ed1ca54548b5ab001e687ad682]) successfully deleted
I20260810 18:52:26.878749 9290 ts_tablet_manager.cc:1905] T c3ced64f204d4a62b80e705a9c234136 P f01b020d35db4744bd395053de3200c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:52:26.880937 9290 ts_tablet_manager.cc:1918] T c3ced64f204d4a62b80e705a9c234136 P f01b020d35db4744bd395053de3200c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:52:26.881054 9290 log.cc:1196] T c3ced64f204d4a62b80e705a9c234136 P f01b020d35db4744bd395053de3200c8: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387687857264-19106-0/minicluster-data/ts-2/wal/wals/c3ced64f204d4a62b80e705a9c234136
I20260810 18:52:26.881443 9290 ts_tablet_manager.cc:1939] T c3ced64f204d4a62b80e705a9c234136 P f01b020d35db4744bd395053de3200c8: Deleting consensus metadata
W20260810 18:52:26.881683 8715 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.168.129:39527 (error 108)
I20260810 18:52:26.881719 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 8949
I20260810 18:52:26.883436 9256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:26.883163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:39527 (local address 127.0.0.1:51976)
0810 18:52:26.883295 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:26.883367 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.129:39527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260810 18:52:26.883577 8715 catalog_manager.cc:4584] TS 79c24fe63325436b8dd0fb2b499e4853 (127.18.168.129:39527): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c3ced64f204d4a62b80e705a9c234136: Network error: Client connection negotiation failed: client connection to 127.18.168.129:39527: connect: Connection refused (error 111)
I20260810 18:52:26.885130 8714 catalog_manager.cc:4841] TS f01b020d35db4744bd395053de3200c8 (127.18.168.131:44709): tablet c3ced64f204d4a62b80e705a9c234136 (table test-table [id=99fc34ed1ca54548b5ab001e687ad682]) successfully deleted
I20260810 18:52:26.890422 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 9101
I20260810 18:52:26.900367 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 8694
2026-08-10T18:52:26Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2406 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-10T18:52:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:52:26Z Disabled control of system clock
I20260810 18:52:26.934103 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:34555
--webserver_interface=127.18.168.190
--webserver_port=0
--builtin_ntp_servers=127.18.168.148:33139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:34555
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 18:52:27.049217 9297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:27.049469 9297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:27.049503 9297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:27.053437 9297 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:52:27.053489 9297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:27.053505 9297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:27.053521 9297 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:52:27.053536 9297 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:52:27.058156 9297 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:33139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:34555
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:34555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:27.059170 9297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:27.060518 9297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:27.069202 9304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:27.069208 9305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:27.069214 9307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:27.069625 9297 server_base.cc:1034] running on GCE node
I20260810 18:52:27.070148 9297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:27.071231 9297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:27.072391 9297 hybrid_clock.cc:648] HybridClock initialized: now 1786387947072366 us; error 44 us; skew 500 ppm
I20260810 18:52:27.074285 9297 webserver.cc:460] Webserver started at http://127.18.168.190:42467/ using document root <none> and password file <none>
I20260810 18:52:27.074831 9297 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:27.074887 9297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:27.075114 9297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:27.077208 9297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "1e3363a6ceb1496d9dccec4f15469733"
format_stamp: "Formatted at 2026-08-10 18:52:27 on dist-test-slave-cbsf"
server_key: "6b7a4093c702c1b738e5e64ff2110a40"
server_key_iv: "3c1a9953d42902a13782e3984236c6ad"
server_key_version: "encryptionkey@0"
I20260810 18:52:27.077772 9297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "1e3363a6ceb1496d9dccec4f15469733"
format_stamp: "Formatted at 2026-08-10 18:52:27 on dist-test-slave-cbsf"
server_key: "6b7a4093c702c1b738e5e64ff2110a40"
server_key_iv: "3c1a9953d42902a13782e3984236c6ad"
server_key_version: "encryptionkey@0"
I20260810 18:52:27.081415 9297 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 18:52:27.083798 9313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:27.085093 9297 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:52:27.085217 9297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "1e3363a6ceb1496d9dccec4f15469733"
format_stamp: "Formatted at 2026-08-10 18:52:27 on dist-test-slave-cbsf"
server_key: "6b7a4093c702c1b738e5e64ff2110a40"
server_key_iv: "3c1a9953d42902a13782e3984236c6ad"
server_key_version: "encryptionkey@0"
I20260810 18:52:27.085325 9297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:27.097903 9297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:27.098495 9297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:27.098737 9297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:27.106006 9297 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:34555
I20260810 18:52:27.106022 9371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:34555 every 8 connection(s)
I20260810 18:52:27.106832 9297 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:52:27.109699 9372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:27.112195 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 9297
I20260810 18:52:27.112349 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:52:27.112628 19106 external_mini_cluster.cc:1428] Setting key 41506ab9ed28eb9d12cfcc65d83b206a
I20260810 18:52:27.115446 9372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e3363a6ceb1496d9dccec4f15469733: Bootstrap starting.
I20260810 18:52:27.117605 9372 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e3363a6ceb1496d9dccec4f15469733: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:27.118323 9372 log.cc:824] T 00000000000000000000000000000000 P 1e3363a6ceb1496d9dccec4f15469733: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:27.120038 9372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e3363a6ceb1496d9dccec4f15469733: No bootstrap required, opened a new log
I20260810 18:52:27.121541 9375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:27.113974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35318 (local address 127.18.168.190:34555)
0810 18:52:27.114573 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:27.114586 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:27.114625 (+ 39us) server_negotiation.cc:407] Connection header received
0810 18:52:27.115159 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:27.115174 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:27.115512 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:27.115859 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:27.116281 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:27.117299 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:27.118235 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:27.118740 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:27.118905 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:27.119727 (+ 822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:27.119732 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:27.119743 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:52:27.120108 (+ 365us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:27.120227 (+ 119us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:52:27.120259 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:27.120355 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:27.120571 (+ 216us) server_negotiation.cc:299] Negotiation successful
0810 18:52:27.120893 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":136,"threads_started":1}
I20260810 18:52:27.123982 9372 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e3363a6ceb1496d9dccec4f15469733 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3363a6ceb1496d9dccec4f15469733" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 34555 } }
I20260810 18:52:27.124199 9372 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e3363a6ceb1496d9dccec4f15469733 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:27.124276 9372 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e3363a6ceb1496d9dccec4f15469733 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e3363a6ceb1496d9dccec4f15469733, State: Initialized, Role: FOLLOWER
I20260810 18:52:27.124725 9372 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e3363a6ceb1496d9dccec4f15469733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3363a6ceb1496d9dccec4f15469733" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 34555 } }
I20260810 18:52:27.124879 9372 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e3363a6ceb1496d9dccec4f15469733 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:52:27.124951 9372 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e3363a6ceb1496d9dccec4f15469733 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:52:27.125044 9372 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e3363a6ceb1496d9dccec4f15469733 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:27.125893 9372 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e3363a6ceb1496d9dccec4f15469733 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3363a6ceb1496d9dccec4f15469733" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 34555 } }
I20260810 18:52:27.126222 9372 leader_election.cc:302] T 00000000000000000000000000000000 P 1e3363a6ceb1496d9dccec4f15469733 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e3363a6ceb1496d9dccec4f15469733; no voters:
I20260810 18:52:27.126497 9372 leader_election.cc:288] T 00000000000000000000000000000000 P 1e3363a6ceb1496d9dccec4f15469733 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:52:27.126598 9377 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e3363a6ceb1496d9dccec4f15469733 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:52:27.126833 9377 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e3363a6ceb1496d9dccec4f15469733 [term 1 LEADER]: Becoming Leader. State: Replica: 1e3363a6ceb1496d9dccec4f15469733, State: Running, Role: LEADER
I20260810 18:52:27.127182 9377 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e3363a6ceb1496d9dccec4f15469733 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3363a6ceb1496d9dccec4f15469733" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 34555 } }
I20260810 18:52:27.127630 9372 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e3363a6ceb1496d9dccec4f15469733 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:52:27.128887 9380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e3363a6ceb1496d9dccec4f15469733 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e3363a6ceb1496d9dccec4f15469733. Latest consensus state: current_term: 1 leader_uuid: "1e3363a6ceb1496d9dccec4f15469733" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3363a6ceb1496d9dccec4f15469733" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 34555 } } }
I20260810 18:52:27.128888 9378 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e3363a6ceb1496d9dccec4f15469733 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e3363a6ceb1496d9dccec4f15469733" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3363a6ceb1496d9dccec4f15469733" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 34555 } } }
I20260810 18:52:27.129026 9378 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e3363a6ceb1496d9dccec4f15469733 [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:27.129025 9380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e3363a6ceb1496d9dccec4f15469733 [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:27.129510 9390 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:52:27.132244 9390 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:52:27.137284 9390 catalog_manager.cc:1347] Generated new cluster ID: 7c4e1c0c2bae4cd0b63dd16c8ccf4a40
I20260810 18:52:27.137346 9390 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:52:27.147082 9390 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:52:27.148159 9390 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:52:27.166760 9390 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e3363a6ceb1496d9dccec4f15469733: Generated new TSK 0
I20260810 18:52:27.167445 9390 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:52:27.174394 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.129:0
--local_ip_for_outbound_sockets=127.18.168.129
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:34555
--builtin_ntp_servers=127.18.168.148:33139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 18:52:27.287106 9401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:27.287318 9401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:27.287350 9401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:27.291399 9401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:27.291448 9401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:27.291553 9401 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.129
I20260810 18:52:27.296175 9401 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:33139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.168.129
--webserver_port=0
--tserver_master_addrs=127.18.168.190:34555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.129
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:27.297254 9401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:27.298637 9401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:27.308156 9410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:27.308172 9408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:27.308177 9406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:27.308543 9401 server_base.cc:1034] running on GCE node
I20260810 18:52:27.308974 9401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:27.309674 9401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:27.310835 9401 hybrid_clock.cc:648] HybridClock initialized: now 1786387947310822 us; error 24 us; skew 500 ppm
I20260810 18:52:27.316807 9401 webserver.cc:460] Webserver started at http://127.18.168.129:44583/ using document root <none> and password file <none>
I20260810 18:52:27.317413 9401 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:27.317502 9401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:27.317721 9401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:27.319821 9401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-0/data/instance:
uuid: "aaab305391974aa4ba455b6e8d51f354"
format_stamp: "Formatted at 2026-08-10 18:52:27 on dist-test-slave-cbsf"
server_key: "d352393d4ec368fa3b146b5f65d0f58c"
server_key_iv: "089be95ac3b8f3694087c32c90dbc219"
server_key_version: "encryptionkey@0"
I20260810 18:52:27.320398 9401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance:
uuid: "aaab305391974aa4ba455b6e8d51f354"
format_stamp: "Formatted at 2026-08-10 18:52:27 on dist-test-slave-cbsf"
server_key: "d352393d4ec368fa3b146b5f65d0f58c"
server_key_iv: "089be95ac3b8f3694087c32c90dbc219"
server_key_version: "encryptionkey@0"
I20260810 18:52:27.324098 9401 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 18:52:27.326542 9417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:27.327633 9401 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 18:52:27.327770 9401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-0/wal
uuid: "aaab305391974aa4ba455b6e8d51f354"
format_stamp: "Formatted at 2026-08-10 18:52:27 on dist-test-slave-cbsf"
server_key: "d352393d4ec368fa3b146b5f65d0f58c"
server_key_iv: "089be95ac3b8f3694087c32c90dbc219"
server_key_version: "encryptionkey@0"
I20260810 18:52:27.327907 9401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:27.346995 9401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:27.347679 9401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:27.347884 9401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:27.348502 9401 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:52:27.349488 9401 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:52:27.349536 9401 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:27.349604 9401 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:52:27.349635 9401 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:27.358553 9401 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.129:36511
I20260810 18:52:27.358613 9545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.129:36511 every 8 connection(s)
I20260810 18:52:27.359440 9401 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-0/data/info.pb
I20260810 18:52:27.361640 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 9401
I20260810 18:52:27.361784 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-0/wal/instance
I20260810 18:52:27.362157 19106 external_mini_cluster.cc:1428] Setting key f978131764e942d0113e41754ffadfa6
I20260810 18:52:27.364343 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.130:0
--local_ip_for_outbound_sockets=127.18.168.130
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:34555
--builtin_ntp_servers=127.18.168.148:33139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 18:52:27.369540 9375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:27.361259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:39381 (local address 127.18.168.190:34555)
0810 18:52:27.361443 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:27.361449 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:27.362648 (+ 1199us) server_negotiation.cc:407] Connection header received
0810 18:52:27.364491 (+ 1843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:27.364496 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:27.364555 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:27.364644 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:27.365818 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:27.366342 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:27.367452 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:27.367691 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:27.367802 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:27.369048 (+ 1246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:27.369052 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:27.369053 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:52:27.369065 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:27.369084 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:52:27.369093 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:27.369173 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:27.369380 (+ 207us) server_negotiation.cc:299] Negotiation successful
0810 18:52:27.369431 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 18:52:27.370676 9548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:27.361667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:34555 (local address 127.18.168.129:39381)
0810 18:52:27.362475 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:27.362530 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:27.364235 (+ 1705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:27.364840 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:27.364852 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:27.364966 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:52:27.365647 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:27.365687 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:27.366481 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:27.366485 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:27.367335 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:27.367344 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:27.367844 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:27.367847 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:27.368066 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:27.368731 (+ 665us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:52:27.368735 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:52:27.368788 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:52:27.368868 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:52:27.369198 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:52:27.369210 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:52:27.369218 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 18:52:27.369340 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 18:52:27.369648 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":127,"threads_started":1}
I20260810 18:52:27.372004 9546 heartbeater.cc:347] Connected to a master server at 127.18.168.190:34555
I20260810 18:52:27.372251 9546 heartbeater.cc:464] Registering TS with master...
I20260810 18:52:27.372823 9546 heartbeater.cc:511] Master 127.18.168.190:34555 requested a full tablet report, sending...
I20260810 18:52:27.374346 9330 ts_manager.cc:194] Registered new tserver with Master: aaab305391974aa4ba455b6e8d51f354 (127.18.168.129:36511)
I20260810 18:52:27.375489 9330 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.168.129:39381
W20260810 18:52:27.476833 9550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:27.477092 9550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:27.477174 9550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:27.481122 9550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:27.481191 9550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:27.481271 9550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.130
I20260810 18:52:27.485891 9550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:33139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.168.130
--webserver_port=0
--tserver_master_addrs=127.18.168.190:34555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.130
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:27.486932 9550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:27.488241 9550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:27.498206 9561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:27.498180 9558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:27.498252 9550 server_base.cc:1034] running on GCE node
W20260810 18:52:27.498169 9559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:27.498888 9550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:27.499517 9550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:27.500679 9550 hybrid_clock.cc:648] HybridClock initialized: now 1786387947500666 us; error 22 us; skew 500 ppm
I20260810 18:52:27.502668 9550 webserver.cc:460] Webserver started at http://127.18.168.130:42551/ using document root <none> and password file <none>
I20260810 18:52:27.503265 9550 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:27.503330 9550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:27.503592 9550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:27.505693 9550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-1/data/instance:
uuid: "876ade6220df4f7f8e0000b5dcf44852"
format_stamp: "Formatted at 2026-08-10 18:52:27 on dist-test-slave-cbsf"
server_key: "ff06175b351fb0c0acf9d689f132b90d"
server_key_iv: "b5be4eba9dc74b8598db453970d8283a"
server_key_version: "encryptionkey@0"
I20260810 18:52:27.506283 9550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance:
uuid: "876ade6220df4f7f8e0000b5dcf44852"
format_stamp: "Formatted at 2026-08-10 18:52:27 on dist-test-slave-cbsf"
server_key: "ff06175b351fb0c0acf9d689f132b90d"
server_key_iv: "b5be4eba9dc74b8598db453970d8283a"
server_key_version: "encryptionkey@0"
I20260810 18:52:27.509850 9550 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:52:27.512395 9567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:27.513640 9550 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 18:52:27.513773 9550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-1/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-1/wal
uuid: "876ade6220df4f7f8e0000b5dcf44852"
format_stamp: "Formatted at 2026-08-10 18:52:27 on dist-test-slave-cbsf"
server_key: "ff06175b351fb0c0acf9d689f132b90d"
server_key_iv: "b5be4eba9dc74b8598db453970d8283a"
server_key_version: "encryptionkey@0"
I20260810 18:52:27.513880 9550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:27.529273 9550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:27.530025 9550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:27.530220 9550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:27.530793 9550 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:52:27.531793 9550 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:52:27.531865 9550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:27.531942 9550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:52:27.531994 9550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:27.540628 9550 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.130:46225
I20260810 18:52:27.540737 9706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.130:46225 every 8 connection(s)
I20260810 18:52:27.541536 9550 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-1/data/info.pb
I20260810 18:52:27.542541 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 9550
I20260810 18:52:27.542677 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-1/wal/instance
I20260810 18:52:27.543031 19106 external_mini_cluster.cc:1428] Setting key d52c3d711f359aea86d3fca3db189327
I20260810 18:52:27.545308 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.168.131:0
--local_ip_for_outbound_sockets=127.18.168.131
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:34555
--builtin_ntp_servers=127.18.168.148:33139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 18:52:27.551638 9375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:27.543479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.130:55557 (local address 127.18.168.190:34555)
0810 18:52:27.543653 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:27.543658 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:27.544927 (+ 1269us) server_negotiation.cc:407] Connection header received
0810 18:52:27.546893 (+ 1966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:27.546895 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:27.546943 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:27.547007 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:27.547947 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:27.548502 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:27.549701 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:27.549939 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:27.550054 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:27.551132 (+ 1078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:27.551135 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:27.551138 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:52:27.551150 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:27.551180 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:52:27.551195 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:27.551274 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:27.551434 (+ 160us) server_negotiation.cc:299] Negotiation successful
0810 18:52:27.551495 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 18:52:27.552438 9711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:27.543973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:34555 (local address 127.18.168.130:55557)
0810 18:52:27.544769 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:27.544823 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:27.546676 (+ 1853us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:27.547196 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:27.547208 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:27.547316 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:52:27.547809 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:27.547822 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:27.548651 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:27.548655 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:27.549552 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:27.549562 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:27.550095 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:27.550099 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:27.550267 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:27.550872 (+ 605us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:52:27.550873 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:52:27.550912 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:52:27.550973 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:52:27.551289 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:52:27.551301 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:52:27.551311 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 18:52:27.551432 (+ 121us) client_negotiation.cc:240] Negotiation successful
0810 18:52:27.551709 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":133,"threads_started":1}
I20260810 18:52:27.553573 9709 heartbeater.cc:347] Connected to a master server at 127.18.168.190:34555
I20260810 18:52:27.553814 9709 heartbeater.cc:464] Registering TS with master...
I20260810 18:52:27.554307 9709 heartbeater.cc:511] Master 127.18.168.190:34555 requested a full tablet report, sending...
I20260810 18:52:27.555356 9330 ts_manager.cc:194] Registered new tserver with Master: 876ade6220df4f7f8e0000b5dcf44852 (127.18.168.130:46225)
I20260810 18:52:27.556226 9330 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.168.130:55557
W20260810 18:52:27.657557 9712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:27.657778 9712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:27.657812 9712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:27.661528 9712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:27.661566 9712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:27.661615 9712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.168.131
I20260810 18:52:27.665910 9712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:33139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.168.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.168.131
--webserver_port=0
--tserver_master_addrs=127.18.168.190:34555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.168.131
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:27.667133 9712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:27.668394 9712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:27.681813 9718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:27.681851 9717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:27.681964 9712 server_base.cc:1034] running on GCE node
W20260810 18:52:27.682103 9723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:27.682582 9712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:27.683405 9712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:27.684590 9712 hybrid_clock.cc:648] HybridClock initialized: now 1786387947684572 us; error 36 us; skew 500 ppm
I20260810 18:52:27.686935 9712 webserver.cc:460] Webserver started at http://127.18.168.131:43509/ using document root <none> and password file <none>
I20260810 18:52:27.687659 9712 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:27.687747 9712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:27.687978 9712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:27.689999 9712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-2/data/instance:
uuid: "3c12691a2f4c4811bdc545f9c10082da"
format_stamp: "Formatted at 2026-08-10 18:52:27 on dist-test-slave-cbsf"
server_key: "7a5dfaf4ab86f5a9461ee36bdb8d6bd4"
server_key_iv: "21947f4a5fbd0a2621d76f0744810f3c"
server_key_version: "encryptionkey@0"
I20260810 18:52:27.690575 9712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance:
uuid: "3c12691a2f4c4811bdc545f9c10082da"
format_stamp: "Formatted at 2026-08-10 18:52:27 on dist-test-slave-cbsf"
server_key: "7a5dfaf4ab86f5a9461ee36bdb8d6bd4"
server_key_iv: "21947f4a5fbd0a2621d76f0744810f3c"
server_key_version: "encryptionkey@0"
I20260810 18:52:27.694078 9712 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:52:27.696597 9731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:27.697868 9712 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:52:27.697981 9712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-2/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-2/wal
uuid: "3c12691a2f4c4811bdc545f9c10082da"
format_stamp: "Formatted at 2026-08-10 18:52:27 on dist-test-slave-cbsf"
server_key: "7a5dfaf4ab86f5a9461ee36bdb8d6bd4"
server_key_iv: "21947f4a5fbd0a2621d76f0744810f3c"
server_key_version: "encryptionkey@0"
I20260810 18:52:27.698060 9712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:27.707454 9712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:27.707995 9712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:27.708127 9712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:27.708627 9712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:52:27.709509 9712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:52:27.709554 9712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:27.709591 9712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:52:27.709613 9712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:27.718547 9712 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.131:42767
I20260810 18:52:27.718688 9859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.131:42767 every 8 connection(s)
I20260810 18:52:27.719414 9712 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-2/data/info.pb
I20260810 18:52:27.722525 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 9712
I20260810 18:52:27.722666 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-2/wal/instance
I20260810 18:52:27.722950 19106 external_mini_cluster.cc:1428] Setting key 5077d0de81acdf836c34c941f1a741fe
I20260810 18:52:27.728610 9375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:27.721081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.131:46819 (local address 127.18.168.190:34555)
0810 18:52:27.721237 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:27.721242 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:27.722274 (+ 1032us) server_negotiation.cc:407] Connection header received
0810 18:52:27.724009 (+ 1735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:27.724011 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:27.724056 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:27.724119 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:27.725232 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:27.725764 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:27.726805 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:27.727020 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:27.727092 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:27.728103 (+ 1011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:27.728115 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:27.728117 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:52:27.728130 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:27.728152 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:52:27.728162 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:27.728248 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:27.728418 (+ 170us) server_negotiation.cc:299] Negotiation successful
0810 18:52:27.728474 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 18:52:27.729470 9863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:27.721496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:34555 (local address 127.18.168.131:46819)
0810 18:52:27.722105 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:27.722143 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:27.723792 (+ 1649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:27.724285 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:27.724291 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:27.724367 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:52:27.725097 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:27.725115 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:27.725868 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:27.725871 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:27.726706 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:27.726715 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:27.727139 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:27.727142 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:27.727266 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:27.727799 (+ 533us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:52:27.727800 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:52:27.727836 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:52:27.727892 (+ 56us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:52:27.728252 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:52:27.728262 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:52:27.728268 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 18:52:27.728402 (+ 134us) client_negotiation.cc:240] Negotiation successful
0810 18:52:27.728667 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":120,"threads_started":1}
I20260810 18:52:27.730461 9860 heartbeater.cc:347] Connected to a master server at 127.18.168.190:34555
I20260810 18:52:27.730677 9860 heartbeater.cc:464] Registering TS with master...
I20260810 18:52:27.731180 9860 heartbeater.cc:511] Master 127.18.168.190:34555 requested a full tablet report, sending...
I20260810 18:52:27.732218 9330 ts_manager.cc:194] Registered new tserver with Master: 3c12691a2f4c4811bdc545f9c10082da (127.18.168.131:42767)
I20260810 18:52:27.732880 9330 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.168.131:46819
I20260810 18:52:27.736835 19106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:52:27.742522 9375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:27.738762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35328 (local address 127.18.168.190:34555)
0810 18:52:27.738902 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:27.738907 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:27.739068 (+ 161us) server_negotiation.cc:407] Connection header received
0810 18:52:27.739203 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:27.739211 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:27.739246 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:27.739330 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:27.739608 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:27.740136 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:27.741048 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:27.741263 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:27.741365 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:27.742085 (+ 720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:27.742094 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:27.742096 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:52:27.742112 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:27.742129 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:52:27.742136 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:27.742221 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:27.742342 (+ 121us) server_negotiation.cc:299] Negotiation successful
0810 18:52:27.742391 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 18:52:27.745153 9330 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35328:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:52:27.747967 9330 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:52:27.761735 9875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:27.756052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:36511 (local address 127.0.0.1:46444)
0810 18:52:27.756570 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:27.756623 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:27.756772 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:27.757628 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:27.757637 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:27.757665 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:27.757804 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:27.757814 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:27.759152 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:27.759157 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:27.760166 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:27.760176 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:27.761224 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:27.761228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:27.761335 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:27.761372 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 18:52:27.761490 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 18:52:27.761596 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":138,"threads_started":1}
I20260810 18:52:27.761785 9874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:27.756219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46444 (local address 127.18.168.129:36511)
0810 18:52:27.757066 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:27.757080 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:27.757104 (+ 24us) server_negotiation.cc:407] Connection header received
0810 18:52:27.757223 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:27.757231 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:27.757435 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:27.757953 (+ 518us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:27.758080 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:27.759036 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:27.760302 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:27.761126 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:27.761243 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:27.761412 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:27.761528 (+ 116us) server_negotiation.cc:299] Negotiation successful
0810 18:52:27.761622 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":737,"thread_start_us":96,"threads_started":1}
I20260810 18:52:27.762475 9879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:27.756224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.131:42767 (local address 127.0.0.1:50350)
0810 18:52:27.757593 (+ 1369us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:27.757607 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:27.757745 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:27.758359 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:27.758361 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:27.758379 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:27.758503 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:27.758509 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:27.759853 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:27.759858 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:27.760947 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:27.760957 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:27.762089 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:27.762092 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:27.762187 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:27.762220 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 18:52:27.762283 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 18:52:27.762350 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1257,"thread_start_us":106,"threads_started":1}
I20260810 18:52:27.762543 9876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:27.756694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50350 (local address 127.18.168.131:42767)
0810 18:52:27.757191 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:27.757222 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:27.757684 (+ 462us) server_negotiation.cc:407] Connection header received
0810 18:52:27.757889 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:27.757896 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:27.758060 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:27.758193 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:27.758676 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:27.759761 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:27.761165 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:27.761955 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:27.762099 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:27.762268 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:27.762314 (+ 46us) server_negotiation.cc:299] Negotiation successful
0810 18:52:27.762397 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":156,"threads_started":1}
I20260810 18:52:27.762604 9873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:27.757098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:46225 (local address 127.0.0.1:37468)
0810 18:52:27.757643 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:27.757658 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:27.757815 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:27.758277 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:27.758281 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:27.758298 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:27.758392 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:27.758399 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:27.759813 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:27.759816 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:27.760912 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:27.760924 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:27.762208 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:27.762214 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:27.762299 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:27.762337 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 18:52:27.762404 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 18:52:27.762473 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"mutex_wait_us":4,"thread_start_us":124,"threads_started":1}
I20260810 18:52:27.762974 9878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:27.757013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37468 (local address 127.18.168.130:46225)
0810 18:52:27.757385 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:27.757403 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:27.757740 (+ 337us) server_negotiation.cc:407] Connection header received
0810 18:52:27.757994 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:27.757998 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:27.758141 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:27.758297 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:27.758502 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:27.759687 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:27.761059 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:27.761854 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:27.761984 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:27.762131 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:27.762693 (+ 562us) server_negotiation.cc:299] Negotiation successful
0810 18:52:27.762791 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":133,"threads_started":1}
I20260810 18:52:27.765219 9785 tablet_service.cc:1467] Processing CreateTablet for tablet d818221942354c4d9df541df739b1994 (DEFAULT_TABLE table=test-table [id=464cd2e0ed7a4110ac7d68260f81006b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:52:27.765254 9471 tablet_service.cc:1467] Processing CreateTablet for tablet d818221942354c4d9df541df739b1994 (DEFAULT_TABLE table=test-table [id=464cd2e0ed7a4110ac7d68260f81006b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:52:27.765604 9625 tablet_service.cc:1467] Processing CreateTablet for tablet d818221942354c4d9df541df739b1994 (DEFAULT_TABLE table=test-table [id=464cd2e0ed7a4110ac7d68260f81006b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:52:27.766256 9785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d818221942354c4d9df541df739b1994. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:27.766288 9471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d818221942354c4d9df541df739b1994. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:27.766472 9625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d818221942354c4d9df541df739b1994. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:27.772675 9880 tablet_bootstrap.cc:492] T d818221942354c4d9df541df739b1994 P 3c12691a2f4c4811bdc545f9c10082da: Bootstrap starting.
I20260810 18:52:27.773751 9881 tablet_bootstrap.cc:492] T d818221942354c4d9df541df739b1994 P 876ade6220df4f7f8e0000b5dcf44852: Bootstrap starting.
I20260810 18:52:27.773918 9882 tablet_bootstrap.cc:492] T d818221942354c4d9df541df739b1994 P aaab305391974aa4ba455b6e8d51f354: Bootstrap starting.
I20260810 18:52:27.774808 9880 tablet_bootstrap.cc:654] T d818221942354c4d9df541df739b1994 P 3c12691a2f4c4811bdc545f9c10082da: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:27.775522 9880 log.cc:824] T d818221942354c4d9df541df739b1994 P 3c12691a2f4c4811bdc545f9c10082da: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:27.775842 9882 tablet_bootstrap.cc:654] T d818221942354c4d9df541df739b1994 P aaab305391974aa4ba455b6e8d51f354: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:27.775842 9881 tablet_bootstrap.cc:654] T d818221942354c4d9df541df739b1994 P 876ade6220df4f7f8e0000b5dcf44852: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:27.776655 9881 log.cc:824] T d818221942354c4d9df541df739b1994 P 876ade6220df4f7f8e0000b5dcf44852: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:27.776655 9882 log.cc:824] T d818221942354c4d9df541df739b1994 P aaab305391974aa4ba455b6e8d51f354: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:27.777534 9880 tablet_bootstrap.cc:492] T d818221942354c4d9df541df739b1994 P 3c12691a2f4c4811bdc545f9c10082da: No bootstrap required, opened a new log
I20260810 18:52:27.777737 9880 ts_tablet_manager.cc:1397] T d818221942354c4d9df541df739b1994 P 3c12691a2f4c4811bdc545f9c10082da: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260810 18:52:27.778687 9881 tablet_bootstrap.cc:492] T d818221942354c4d9df541df739b1994 P 876ade6220df4f7f8e0000b5dcf44852: No bootstrap required, opened a new log
I20260810 18:52:27.778864 9881 ts_tablet_manager.cc:1397] T d818221942354c4d9df541df739b1994 P 876ade6220df4f7f8e0000b5dcf44852: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 18:52:27.778947 9882 tablet_bootstrap.cc:492] T d818221942354c4d9df541df739b1994 P aaab305391974aa4ba455b6e8d51f354: No bootstrap required, opened a new log
I20260810 18:52:27.779117 9882 ts_tablet_manager.cc:1397] T d818221942354c4d9df541df739b1994 P aaab305391974aa4ba455b6e8d51f354: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260810 18:52:27.781081 9880 raft_consensus.cc:348] T d818221942354c4d9df541df739b1994 P 3c12691a2f4c4811bdc545f9c10082da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaab305391974aa4ba455b6e8d51f354" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 36511 } } peers { permanent_uuid: "3c12691a2f4c4811bdc545f9c10082da" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 42767 } } peers { permanent_uuid: "876ade6220df4f7f8e0000b5dcf44852" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 46225 } }
I20260810 18:52:27.781322 9880 raft_consensus.cc:374] T d818221942354c4d9df541df739b1994 P 3c12691a2f4c4811bdc545f9c10082da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:27.781394 9880 raft_consensus.cc:729] T d818221942354c4d9df541df739b1994 P 3c12691a2f4c4811bdc545f9c10082da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c12691a2f4c4811bdc545f9c10082da, State: Initialized, Role: FOLLOWER
I20260810 18:52:27.781847 9880 consensus_queue.cc:260] T d818221942354c4d9df541df739b1994 P 3c12691a2f4c4811bdc545f9c10082da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaab305391974aa4ba455b6e8d51f354" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 36511 } } peers { permanent_uuid: "3c12691a2f4c4811bdc545f9c10082da" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 42767 } } peers { permanent_uuid: "876ade6220df4f7f8e0000b5dcf44852" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 46225 } }
I20260810 18:52:27.782480 9860 heartbeater.cc:503] Master 127.18.168.190:34555 was elected leader, sending a full tablet report...
I20260810 18:52:27.782843 9880 ts_tablet_manager.cc:1428] T d818221942354c4d9df541df739b1994 P 3c12691a2f4c4811bdc545f9c10082da: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260810 18:52:27.783097 9881 raft_consensus.cc:348] T d818221942354c4d9df541df739b1994 P 876ade6220df4f7f8e0000b5dcf44852 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaab305391974aa4ba455b6e8d51f354" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 36511 } } peers { permanent_uuid: "3c12691a2f4c4811bdc545f9c10082da" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 42767 } } peers { permanent_uuid: "876ade6220df4f7f8e0000b5dcf44852" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 46225 } }
I20260810 18:52:27.783097 9882 raft_consensus.cc:348] T d818221942354c4d9df541df739b1994 P aaab305391974aa4ba455b6e8d51f354 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaab305391974aa4ba455b6e8d51f354" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 36511 } } peers { permanent_uuid: "3c12691a2f4c4811bdc545f9c10082da" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 42767 } } peers { permanent_uuid: "876ade6220df4f7f8e0000b5dcf44852" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 46225 } }
I20260810 18:52:27.783319 9881 raft_consensus.cc:374] T d818221942354c4d9df541df739b1994 P 876ade6220df4f7f8e0000b5dcf44852 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:27.783319 9882 raft_consensus.cc:374] T d818221942354c4d9df541df739b1994 P aaab305391974aa4ba455b6e8d51f354 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:27.783370 9882 raft_consensus.cc:729] T d818221942354c4d9df541df739b1994 P aaab305391974aa4ba455b6e8d51f354 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aaab305391974aa4ba455b6e8d51f354, State: Initialized, Role: FOLLOWER
I20260810 18:52:27.783370 9881 raft_consensus.cc:729] T d818221942354c4d9df541df739b1994 P 876ade6220df4f7f8e0000b5dcf44852 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 876ade6220df4f7f8e0000b5dcf44852, State: Initialized, Role: FOLLOWER
I20260810 18:52:27.783772 9882 consensus_queue.cc:260] T d818221942354c4d9df541df739b1994 P aaab305391974aa4ba455b6e8d51f354 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaab305391974aa4ba455b6e8d51f354" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 36511 } } peers { permanent_uuid: "3c12691a2f4c4811bdc545f9c10082da" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 42767 } } peers { permanent_uuid: "876ade6220df4f7f8e0000b5dcf44852" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 46225 } }
I20260810 18:52:27.783772 9881 consensus_queue.cc:260] T d818221942354c4d9df541df739b1994 P 876ade6220df4f7f8e0000b5dcf44852 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaab305391974aa4ba455b6e8d51f354" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 36511 } } peers { permanent_uuid: "3c12691a2f4c4811bdc545f9c10082da" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 42767 } } peers { permanent_uuid: "876ade6220df4f7f8e0000b5dcf44852" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 46225 } }
I20260810 18:52:27.784456 9709 heartbeater.cc:503] Master 127.18.168.190:34555 was elected leader, sending a full tablet report...
I20260810 18:52:27.784773 9882 ts_tablet_manager.cc:1428] T d818221942354c4d9df541df739b1994 P aaab305391974aa4ba455b6e8d51f354: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260810 18:52:27.784773 9881 ts_tablet_manager.cc:1428] T d818221942354c4d9df541df739b1994 P 876ade6220df4f7f8e0000b5dcf44852: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260810 18:52:27.785136 9546 heartbeater.cc:503] Master 127.18.168.190:34555 was elected leader, sending a full tablet report...
W20260810 18:52:27.792469 9710 tablet.cc:2369] T d818221942354c4d9df541df739b1994 P 876ade6220df4f7f8e0000b5dcf44852: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:52:27.860709 9547 tablet.cc:2369] T d818221942354c4d9df541df739b1994 P aaab305391974aa4ba455b6e8d51f354: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:52:27.970377 9861 tablet.cc:2369] T d818221942354c4d9df541df739b1994 P 3c12691a2f4c4811bdc545f9c10082da: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:52:28.072786 9888 raft_consensus.cc:482] T d818221942354c4d9df541df739b1994 P aaab305391974aa4ba455b6e8d51f354 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:52:28.073022 9888 raft_consensus.cc:504] T d818221942354c4d9df541df739b1994 P aaab305391974aa4ba455b6e8d51f354 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaab305391974aa4ba455b6e8d51f354" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 36511 } } peers { permanent_uuid: "3c12691a2f4c4811bdc545f9c10082da" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 42767 } } peers { permanent_uuid: "876ade6220df4f7f8e0000b5dcf44852" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 46225 } }
I20260810 18:52:28.074190 9888 leader_election.cc:288] T d818221942354c4d9df541df739b1994 P aaab305391974aa4ba455b6e8d51f354 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c12691a2f4c4811bdc545f9c10082da (127.18.168.131:42767), 876ade6220df4f7f8e0000b5dcf44852 (127.18.168.130:46225)
I20260810 18:52:28.078336 9878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:28.074726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:36557 (local address 127.18.168.130:46225)
0810 18:52:28.074947 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:28.074952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:28.075057 (+ 105us) server_negotiation.cc:407] Connection header received
0810 18:52:28.075195 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:28.075198 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:28.075250 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:28.075307 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:28.075611 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:28.076154 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:28.077336 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:28.077772 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:28.077863 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:28.077902 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:28.078123 (+ 221us) server_negotiation.cc:299] Negotiation successful
0810 18:52:28.078208 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260810 18:52:28.078342 9890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:28.074500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:46225 (local address 127.18.168.129:36557)
0810 18:52:28.074960 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:28.074973 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:28.075083 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:28.075345 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:28.075348 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:28.075363 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:28.075519 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:28.075525 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:28.076255 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:28.076258 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:28.077188 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:28.077198 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:28.077891 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:28.077897 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:28.077992 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:28.078015 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 18:52:28.078093 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 18:52:28.078151 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"mutex_wait_us":57,"spinlock_wait_cycles":26368,"thread_start_us":112,"threads_started":1}
I20260810 18:52:28.078460 9876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:28.074636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.168.129:47087 (local address 127.18.168.131:42767)
0810 18:52:28.074870 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:28.074875 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:28.074971 (+ 96us) server_negotiation.cc:407] Connection header received
0810 18:52:28.075211 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:28.075213 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:28.075258 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:28.075346 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:28.075687 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:28.076319 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:28.077445 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:28.077874 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:28.077986 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:28.078033 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:28.078250 (+ 217us) server_negotiation.cc:299] Negotiation successful
0810 18:52:28.078322 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136}
I20260810 18:52:28.078495 9889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:28.074500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.131:42767 (local address 127.18.168.129:47087)
0810 18:52:28.074841 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:28.074859 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:28.075058 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:28.075379 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:28.075382 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:28.075425 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:28.075564 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:28.075573 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:28.076427 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:28.076430 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:28.077319 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:28.077330 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:28.078006 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:28.078009 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:28.078117 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:28.078142 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 18:52:28.078217 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 18:52:28.078279 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"spinlock_wait_cycles":2688,"thread_start_us":94,"threads_started":1}
I20260810 18:52:28.079110 9648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d818221942354c4d9df541df739b1994" candidate_uuid: "aaab305391974aa4ba455b6e8d51f354" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "876ade6220df4f7f8e0000b5dcf44852" is_pre_election: true
I20260810 18:52:28.079134 9807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d818221942354c4d9df541df739b1994" candidate_uuid: "aaab305391974aa4ba455b6e8d51f354" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c12691a2f4c4811bdc545f9c10082da" is_pre_election: true
I20260810 18:52:28.079416 9807 raft_consensus.cc:2454] T d818221942354c4d9df541df739b1994 P 3c12691a2f4c4811bdc545f9c10082da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aaab305391974aa4ba455b6e8d51f354 in term 0.
I20260810 18:52:28.079437 9648 raft_consensus.cc:2454] T d818221942354c4d9df541df739b1994 P 876ade6220df4f7f8e0000b5dcf44852 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aaab305391974aa4ba455b6e8d51f354 in term 0.
I20260810 18:52:28.079991 9418 leader_election.cc:302] T d818221942354c4d9df541df739b1994 P aaab305391974aa4ba455b6e8d51f354 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3c12691a2f4c4811bdc545f9c10082da, 876ade6220df4f7f8e0000b5dcf44852, aaab305391974aa4ba455b6e8d51f354; no voters:
I20260810 18:52:28.080269 9888 raft_consensus.cc:2781] T d818221942354c4d9df541df739b1994 P aaab305391974aa4ba455b6e8d51f354 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:52:28.080370 9888 raft_consensus.cc:482] T d818221942354c4d9df541df739b1994 P aaab305391974aa4ba455b6e8d51f354 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:52:28.080461 9888 raft_consensus.cc:3037] T d818221942354c4d9df541df739b1994 P aaab305391974aa4ba455b6e8d51f354 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:28.081598 9888 raft_consensus.cc:504] T d818221942354c4d9df541df739b1994 P aaab305391974aa4ba455b6e8d51f354 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaab305391974aa4ba455b6e8d51f354" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 36511 } } peers { permanent_uuid: "3c12691a2f4c4811bdc545f9c10082da" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 42767 } } peers { permanent_uuid: "876ade6220df4f7f8e0000b5dcf44852" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 46225 } }
I20260810 18:52:28.081961 9888 leader_election.cc:288] T d818221942354c4d9df541df739b1994 P aaab305391974aa4ba455b6e8d51f354 [CANDIDATE]: Term 1 election: Requested vote from peers 3c12691a2f4c4811bdc545f9c10082da (127.18.168.131:42767), 876ade6220df4f7f8e0000b5dcf44852 (127.18.168.130:46225)
I20260810 18:52:28.082427 9807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d818221942354c4d9df541df739b1994" candidate_uuid: "aaab305391974aa4ba455b6e8d51f354" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c12691a2f4c4811bdc545f9c10082da"
I20260810 18:52:28.082476 9648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d818221942354c4d9df541df739b1994" candidate_uuid: "aaab305391974aa4ba455b6e8d51f354" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "876ade6220df4f7f8e0000b5dcf44852"
I20260810 18:52:28.082593 9807 raft_consensus.cc:3037] T d818221942354c4d9df541df739b1994 P 3c12691a2f4c4811bdc545f9c10082da [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:28.082598 9648 raft_consensus.cc:3037] T d818221942354c4d9df541df739b1994 P 876ade6220df4f7f8e0000b5dcf44852 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:28.084007 9807 raft_consensus.cc:2454] T d818221942354c4d9df541df739b1994 P 3c12691a2f4c4811bdc545f9c10082da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aaab305391974aa4ba455b6e8d51f354 in term 1.
I20260810 18:52:28.084007 9648 raft_consensus.cc:2454] T d818221942354c4d9df541df739b1994 P 876ade6220df4f7f8e0000b5dcf44852 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aaab305391974aa4ba455b6e8d51f354 in term 1.
I20260810 18:52:28.084374 9420 leader_election.cc:302] T d818221942354c4d9df541df739b1994 P aaab305391974aa4ba455b6e8d51f354 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 876ade6220df4f7f8e0000b5dcf44852, aaab305391974aa4ba455b6e8d51f354; no voters:
I20260810 18:52:28.084611 9888 raft_consensus.cc:2781] T d818221942354c4d9df541df739b1994 P aaab305391974aa4ba455b6e8d51f354 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:52:28.084853 9888 raft_consensus.cc:686] T d818221942354c4d9df541df739b1994 P aaab305391974aa4ba455b6e8d51f354 [term 1 LEADER]: Becoming Leader. State: Replica: aaab305391974aa4ba455b6e8d51f354, State: Running, Role: LEADER
I20260810 18:52:28.085209 9888 consensus_queue.cc:237] T d818221942354c4d9df541df739b1994 P aaab305391974aa4ba455b6e8d51f354 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaab305391974aa4ba455b6e8d51f354" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 36511 } } peers { permanent_uuid: "3c12691a2f4c4811bdc545f9c10082da" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 42767 } } peers { permanent_uuid: "876ade6220df4f7f8e0000b5dcf44852" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 46225 } }
I20260810 18:52:28.088332 9329 catalog_manager.cc:5495] T d818221942354c4d9df541df739b1994 P aaab305391974aa4ba455b6e8d51f354 reported cstate change: term changed from 0 to 1, leader changed from <none> to aaab305391974aa4ba455b6e8d51f354 (127.18.168.129). New cstate: current_term: 1 leader_uuid: "aaab305391974aa4ba455b6e8d51f354" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaab305391974aa4ba455b6e8d51f354" member_type: VOTER last_known_addr { host: "127.18.168.129" port: 36511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c12691a2f4c4811bdc545f9c10082da" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 42767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "876ade6220df4f7f8e0000b5dcf44852" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 46225 } health_report { overall_health: UNKNOWN } } }
I20260810 18:52:28.124936 9874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:28.121245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46456 (local address 127.18.168.129:36511)
0810 18:52:28.121386 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:28.121391 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:28.121407 (+ 16us) server_negotiation.cc:407] Connection header received
0810 18:52:28.121505 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:28.121507 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:28.121564 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:28.121682 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:52:28.121923 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:28.122536 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:28.123420 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:28.123696 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:28.123807 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:28.124120 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:52:28.124509 (+ 389us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:52:28.124664 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:28.124727 (+ 63us) server_negotiation.cc:299] Negotiation successful
0810 18:52:28.124778 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 18:52:28.130723 9648 raft_consensus.cc:1261] T d818221942354c4d9df541df739b1994 P 876ade6220df4f7f8e0000b5dcf44852 [term 1 FOLLOWER]: Refusing update from remote peer aaab305391974aa4ba455b6e8d51f354: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:52:28.130723 9807 raft_consensus.cc:1261] T d818221942354c4d9df541df739b1994 P 3c12691a2f4c4811bdc545f9c10082da [term 1 FOLLOWER]: Refusing update from remote peer aaab305391974aa4ba455b6e8d51f354: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:52:28.131404 9891 consensus_queue.cc:1035] T d818221942354c4d9df541df739b1994 P aaab305391974aa4ba455b6e8d51f354 [LEADER]: Connected to new peer: Peer: permanent_uuid: "876ade6220df4f7f8e0000b5dcf44852" member_type: VOTER last_known_addr { host: "127.18.168.130" port: 46225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:52:28.131611 9888 consensus_queue.cc:1035] T d818221942354c4d9df541df739b1994 P aaab305391974aa4ba455b6e8d51f354 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c12691a2f4c4811bdc545f9c10082da" member_type: VOTER last_known_addr { host: "127.18.168.131" port: 42767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:52:28.147336 9328 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:52:28.147505 9328 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:52:28.149737 9328 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1e3363a6ceb1496d9dccec4f15469733: Sending DeleteTablet for 3 replicas of tablet d818221942354c4d9df541df739b1994
I20260810 18:52:28.150480 9785 tablet_service.cc:1514] Processing DeleteTablet for tablet d818221942354c4d9df541df739b1994 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:52:28 UTC) from {username='slave'} at 127.0.0.1:50350
I20260810 18:52:28.150492 9471 tablet_service.cc:1514] Processing DeleteTablet for tablet d818221942354c4d9df541df739b1994 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:52:28 UTC) from {username='slave'} at 127.0.0.1:46444
I20260810 18:52:28.150619 9625 tablet_service.cc:1514] Processing DeleteTablet for tablet d818221942354c4d9df541df739b1994 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:52:28 UTC) from {username='slave'} at 127.0.0.1:37468
I20260810 18:52:28.150992 9906 tablet_replica.cc:330] T d818221942354c4d9df541df739b1994 P 3c12691a2f4c4811bdc545f9c10082da: stopping tablet replica
I20260810 18:52:28.151310 9906 raft_consensus.cc:2229] T d818221942354c4d9df541df739b1994 P 3c12691a2f4c4811bdc545f9c10082da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:52:28.151669 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 9401
I20260810 18:52:28.151682 9906 raft_consensus.cc:2258] T d818221942354c4d9df541df739b1994 P 3c12691a2f4c4811bdc545f9c10082da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:52:28.155707 9906 ts_tablet_manager.cc:1905] T d818221942354c4d9df541df739b1994 P 3c12691a2f4c4811bdc545f9c10082da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:52:28.158427 9906 ts_tablet_manager.cc:1918] T d818221942354c4d9df541df739b1994 P 3c12691a2f4c4811bdc545f9c10082da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:52:28.158506 9906 log.cc:1196] T d818221942354c4d9df541df739b1994 P 3c12691a2f4c4811bdc545f9c10082da: Deleting WAL directory at /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387687857264-19106-0/minicluster-data/ts-2/wal/wals/d818221942354c4d9df541df739b1994
I20260810 18:52:28.158826 9906 ts_tablet_manager.cc:1939] T d818221942354c4d9df541df739b1994 P 3c12691a2f4c4811bdc545f9c10082da: Deleting consensus metadata
I20260810 18:52:28.160038 9314 catalog_manager.cc:4841] TS 3c12691a2f4c4811bdc545f9c10082da (127.18.168.131:42767): tablet d818221942354c4d9df541df739b1994 (table test-table [id=464cd2e0ed7a4110ac7d68260f81006b]) successfully deleted
I20260810 18:52:28.160925 9908 tablet_replica.cc:330] T d818221942354c4d9df541df739b1994 P 876ade6220df4f7f8e0000b5dcf44852: stopping tablet replica
I20260810 18:52:28.161222 9908 raft_consensus.cc:2229] T d818221942354c4d9df541df739b1994 P 876ade6220df4f7f8e0000b5dcf44852 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:52:28.161520 9908 raft_consensus.cc:2258] T d818221942354c4d9df541df739b1994 P 876ade6220df4f7f8e0000b5dcf44852 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:52:28.162801 9908 ts_tablet_manager.cc:1905] T d818221942354c4d9df541df739b1994 P 876ade6220df4f7f8e0000b5dcf44852: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 18:52:28.163079 9315 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.168.129:36511 (error 108)
I20260810 18:52:28.163146 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 9550
I20260810 18:52:28.165584 9879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:28.165304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.129:36511 (local address 127.0.0.1:46470)
0810 18:52:28.165455 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:28.165514 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.129:36511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 18:52:28.165835 9315 catalog_manager.cc:4584] TS aaab305391974aa4ba455b6e8d51f354 (127.18.168.129:36511): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d818221942354c4d9df541df739b1994: Network error: Client connection negotiation failed: client connection to 127.18.168.129:36511: connect: Connection refused (error 111)
I20260810 18:52:28.171820 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 9712
I20260810 18:52:28.172123 9879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:28.171852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.130:46225 (local address 127.0.0.1:37476)
0810 18:52:28.171980 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:28.172038 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.130:46225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260810 18:52:28.179807 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 9297
2026-08-10T18:52:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1278 ms)
[----------] 6 tests from EncryptionPolicyTest (11087 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:52:28 dist-test-slave-cbsf krb5kdc[9912](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:52:28 dist-test-slave-cbsf krb5kdc[9912](info): set up 2 sockets
Aug 10 18:52:28 dist-test-slave-cbsf krb5kdc[9912](info): commencing operation
krb5kdc: starting...
W20260810 18:52:29.280462 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:52:29 dist-test-slave-cbsf krb5kdc[9912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:52:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:52:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:52:29.405689 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38159
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38159
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:52:29.517051 9926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:29.517330 9926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:29.517373 9926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:29.521371 9926 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:52:29.521430 9926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:29.521456 9926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:29.521473 9926 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:52:29.521488 9926 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:52:29.526695 9926 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38159
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:29.527632 9926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:29.528595 9926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 18:52:29.537168 9926 server_base.cc:1034] running on GCE node
W20260810 18:52:29.537097 9933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:29.537061 9935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:29.537086 9932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:29.538094 9926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:29.539124 9926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:29.540298 9926 hybrid_clock.cc:648] HybridClock initialized: now 1786387949540264 us; error 39 us; skew 500 ppm
Aug 10 18:52:29 dist-test-slave-cbsf krb5kdc[9912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387949, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:29.543133 9926 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:52:29.544004 9926 webserver.cc:460] Webserver started at http://127.0.0.1:36553/ using document root <none> and password file <none>
I20260810 18:52:29.544629 9926 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:29.544754 9926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:29.544994 9926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:29.546984 9926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "f430b45130e64ebba9db3c15d3e45a1e"
format_stamp: "Formatted at 2026-08-10 18:52:29 on dist-test-slave-cbsf"
server_key: "1dc186ebc5243776e3bed588c892861d"
server_key_iv: "348250dfe54ce940ccba2a1dff4df8c0"
server_key_version: "encryptionkey@0"
I20260810 18:52:29.547592 9926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "f430b45130e64ebba9db3c15d3e45a1e"
format_stamp: "Formatted at 2026-08-10 18:52:29 on dist-test-slave-cbsf"
server_key: "1dc186ebc5243776e3bed588c892861d"
server_key_iv: "348250dfe54ce940ccba2a1dff4df8c0"
server_key_version: "encryptionkey@0"
I20260810 18:52:29.551486 9926 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:52:29.554072 9945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:29.555315 9926 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:52:29.555490 9926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "f430b45130e64ebba9db3c15d3e45a1e"
format_stamp: "Formatted at 2026-08-10 18:52:29 on dist-test-slave-cbsf"
server_key: "1dc186ebc5243776e3bed588c892861d"
server_key_iv: "348250dfe54ce940ccba2a1dff4df8c0"
server_key_version: "encryptionkey@0"
I20260810 18:52:29.555606 9926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:29.573576 9926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:29.576169 9926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:29.576385 9926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:29.583813 9926 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38159
I20260810 18:52:29.583832 10002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38159 every 8 connection(s)
I20260810 18:52:29.584792 9926 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:52:29.587602 10003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:29.591883 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 9926
I20260810 18:52:29.592031 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:52:29.592336 19106 external_mini_cluster.cc:1428] Setting key 37ebacc1ef0e1d5cc994ffa2e2b8ac37
I20260810 18:52:29.593590 10003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f430b45130e64ebba9db3c15d3e45a1e: Bootstrap starting.
I20260810 18:52:29.595836 10003 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f430b45130e64ebba9db3c15d3e45a1e: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:29.596550 10003 log.cc:824] T 00000000000000000000000000000000 P f430b45130e64ebba9db3c15d3e45a1e: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:29.598415 10003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f430b45130e64ebba9db3c15d3e45a1e: No bootstrap required, opened a new log
Aug 10 18:52:29 dist-test-slave-cbsf krb5kdc[9912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 18:52:29.602210 10003 raft_consensus.cc:348] T 00000000000000000000000000000000 P f430b45130e64ebba9db3c15d3e45a1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f430b45130e64ebba9db3c15d3e45a1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38159 } }
I20260810 18:52:29.602456 10003 raft_consensus.cc:374] T 00000000000000000000000000000000 P f430b45130e64ebba9db3c15d3e45a1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:29.602574 10003 raft_consensus.cc:729] T 00000000000000000000000000000000 P f430b45130e64ebba9db3c15d3e45a1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f430b45130e64ebba9db3c15d3e45a1e, State: Initialized, Role: FOLLOWER
I20260810 18:52:29.603024 10003 consensus_queue.cc:260] T 00000000000000000000000000000000 P f430b45130e64ebba9db3c15d3e45a1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f430b45130e64ebba9db3c15d3e45a1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38159 } }
I20260810 18:52:29.603179 10003 raft_consensus.cc:388] T 00000000000000000000000000000000 P f430b45130e64ebba9db3c15d3e45a1e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:52:29.603276 10003 raft_consensus.cc:482] T 00000000000000000000000000000000 P f430b45130e64ebba9db3c15d3e45a1e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:52:29.603399 10003 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f430b45130e64ebba9db3c15d3e45a1e [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:29.604240 10003 raft_consensus.cc:504] T 00000000000000000000000000000000 P f430b45130e64ebba9db3c15d3e45a1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f430b45130e64ebba9db3c15d3e45a1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38159 } }
I20260810 18:52:29.604595 10003 leader_election.cc:302] T 00000000000000000000000000000000 P f430b45130e64ebba9db3c15d3e45a1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f430b45130e64ebba9db3c15d3e45a1e; no voters:
I20260810 18:52:29.604936 10003 leader_election.cc:288] T 00000000000000000000000000000000 P f430b45130e64ebba9db3c15d3e45a1e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:52:29.605014 10010 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f430b45130e64ebba9db3c15d3e45a1e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:52:29.605261 10010 raft_consensus.cc:686] T 00000000000000000000000000000000 P f430b45130e64ebba9db3c15d3e45a1e [term 1 LEADER]: Becoming Leader. State: Replica: f430b45130e64ebba9db3c15d3e45a1e, State: Running, Role: LEADER
I20260810 18:52:29.605656 10010 consensus_queue.cc:237] T 00000000000000000000000000000000 P f430b45130e64ebba9db3c15d3e45a1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f430b45130e64ebba9db3c15d3e45a1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38159 } }
I20260810 18:52:29.606273 10003 sys_catalog.cc:564] T 00000000000000000000000000000000 P f430b45130e64ebba9db3c15d3e45a1e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:52:29.607048 10010 sys_catalog.cc:455] T 00000000000000000000000000000000 P f430b45130e64ebba9db3c15d3e45a1e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f430b45130e64ebba9db3c15d3e45a1e. Latest consensus state: current_term: 1 leader_uuid: "f430b45130e64ebba9db3c15d3e45a1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f430b45130e64ebba9db3c15d3e45a1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38159 } } }
I20260810 18:52:29.607158 10010 sys_catalog.cc:458] T 00000000000000000000000000000000 P f430b45130e64ebba9db3c15d3e45a1e [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:29.607395 10006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:29.593874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35169 (local address 127.0.0.1:38159)
0810 18:52:29.594315 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:29.594331 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:29.594365 (+ 34us) server_negotiation.cc:407] Connection header received
0810 18:52:29.594968 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:29.594983 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:29.595309 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:29.595621 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:29.596427 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:29.597487 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:29.598358 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:29.598895 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:29.599059 (+ 164us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:29.601579 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:29.601584 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:29.601597 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:29.601624 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:29.603549 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:29.604053 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:29.604059 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:29.604065 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:29.604184 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:29.604583 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:29.604586 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:29.604588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:29.605181 (+ 593us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:29.605976 (+ 795us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:29.606164 (+ 188us) server_negotiation.cc:299] Negotiation successful
0810 18:52:29.606404 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":140,"threads_started":1}
I20260810 18:52:29.607508 10018 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:52:29.608742 10011 sys_catalog.cc:455] T 00000000000000000000000000000000 P f430b45130e64ebba9db3c15d3e45a1e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f430b45130e64ebba9db3c15d3e45a1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f430b45130e64ebba9db3c15d3e45a1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38159 } } }
I20260810 18:52:29.608872 10011 sys_catalog.cc:458] T 00000000000000000000000000000000 P f430b45130e64ebba9db3c15d3e45a1e [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:29.610553 19106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:52:29.610698 10018 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:52:29.616557 10018 catalog_manager.cc:1347] Generated new cluster ID: 369d59479e314871a0fd248d8574800a
I20260810 18:52:29.616657 10018 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:52:29.622196 10006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:29.613110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60219 (local address 127.0.0.1:38159)
0810 18:52:29.613208 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:29.613212 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:29.613258 (+ 46us) server_negotiation.cc:407] Connection header received
0810 18:52:29.613399 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:29.613402 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:29.613442 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:29.613519 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:29.614369 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:29.614992 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:29.616081 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:29.617030 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:29.617141 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:29.618679 (+ 1538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:29.618683 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:29.618685 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:29.618710 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:29.620316 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:29.620841 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:29.620844 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:29.620846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:29.620931 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:29.621280 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:29.621284 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:29.621286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:29.621505 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:29.621658 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:29.621924 (+ 266us) server_negotiation.cc:299] Negotiation successful
0810 18:52:29.622038 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 18:52:29.624598 10018 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:52:29.625816 10018 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:52:29.632030 10018 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f430b45130e64ebba9db3c15d3e45a1e: Generated new TSK 0
I20260810 18:52:29.632813 10018 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:52:29.640486 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 9926
W20260810 18:52:29.651607 9910 connection.cc:535] client connection to 127.0.0.1:38159 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38159): Connection reset by peer (error 104)
2026-08-10T18:52:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1462 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:52:29 dist-test-slave-cbsf krb5kdc[10040](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:52:29 dist-test-slave-cbsf krb5kdc[10040](info): set up 2 sockets
Aug 10 18:52:29 dist-test-slave-cbsf krb5kdc[10040](info): commencing operation
krb5kdc: starting...
W20260810 18:52:30.754922 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:52:30 dist-test-slave-cbsf krb5kdc[10040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:52:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:52:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:52:30.894908 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46547
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46547
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:52:31.010641 10055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:31.010905 10055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:31.010983 10055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:31.014912 10055 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:52:31.014989 10055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:31.015026 10055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:31.015082 10055 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:52:31.015120 10055 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:52:31.020213 10055 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46547
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:31.021236 10055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:31.022250 10055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:31.030783 10063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:31.030768 10061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:31.030798 10060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:31.031071 10055 server_base.cc:1034] running on GCE node
I20260810 18:52:31.031579 10055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:31.032757 10055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:31.033995 10055 hybrid_clock.cc:648] HybridClock initialized: now 1786387951033971 us; error 27 us; skew 500 ppm
Aug 10 18:52:31 dist-test-slave-cbsf krb5kdc[10040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387951, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:31.036993 10055 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:52:31.037889 10055 webserver.cc:460] Webserver started at http://127.0.0.1:33645/ using document root <none> and password file <none>
I20260810 18:52:31.038633 10055 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:31.038754 10055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:31.039011 10055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:31.041085 10055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "dca3eb6a54264cfe82820f4fbf82e390"
format_stamp: "Formatted at 2026-08-10 18:52:31 on dist-test-slave-cbsf"
server_key: "da981eebc877856706b0756733b479cd"
server_key_iv: "bd7285249853dc164adcaa88222d1007"
server_key_version: "encryptionkey@0"
I20260810 18:52:31.041730 10055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "dca3eb6a54264cfe82820f4fbf82e390"
format_stamp: "Formatted at 2026-08-10 18:52:31 on dist-test-slave-cbsf"
server_key: "da981eebc877856706b0756733b479cd"
server_key_iv: "bd7285249853dc164adcaa88222d1007"
server_key_version: "encryptionkey@0"
I20260810 18:52:31.045440 10055 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:52:31.047727 10070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:31.048923 10055 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:52:31.049062 10055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "dca3eb6a54264cfe82820f4fbf82e390"
format_stamp: "Formatted at 2026-08-10 18:52:31 on dist-test-slave-cbsf"
server_key: "da981eebc877856706b0756733b479cd"
server_key_iv: "bd7285249853dc164adcaa88222d1007"
server_key_version: "encryptionkey@0"
I20260810 18:52:31.049173 10055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:31.068046 10055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:31.071568 10055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:31.071794 10055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:31.079990 10055 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46547
I20260810 18:52:31.080009 10133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46547 every 8 connection(s)
I20260810 18:52:31.081132 10055 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:52:31.083993 10134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:31.089359 10134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dca3eb6a54264cfe82820f4fbf82e390: Bootstrap starting.
I20260810 18:52:31.090794 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 10055
I20260810 18:52:31.090932 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:52:31.091252 19106 external_mini_cluster.cc:1428] Setting key f0b234c1e25daf4d2c9a5f4d199e53e7
I20260810 18:52:31.092212 10134 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dca3eb6a54264cfe82820f4fbf82e390: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:31.093116 10134 log.cc:824] T 00000000000000000000000000000000 P dca3eb6a54264cfe82820f4fbf82e390: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:31.095068 10134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dca3eb6a54264cfe82820f4fbf82e390: No bootstrap required, opened a new log
I20260810 18:52:31.098487 10134 raft_consensus.cc:348] T 00000000000000000000000000000000 P dca3eb6a54264cfe82820f4fbf82e390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dca3eb6a54264cfe82820f4fbf82e390" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46547 } }
I20260810 18:52:31.098686 10134 raft_consensus.cc:374] T 00000000000000000000000000000000 P dca3eb6a54264cfe82820f4fbf82e390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:31.098744 10134 raft_consensus.cc:729] T 00000000000000000000000000000000 P dca3eb6a54264cfe82820f4fbf82e390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dca3eb6a54264cfe82820f4fbf82e390, State: Initialized, Role: FOLLOWER
I20260810 18:52:31.099190 10134 consensus_queue.cc:260] T 00000000000000000000000000000000 P dca3eb6a54264cfe82820f4fbf82e390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dca3eb6a54264cfe82820f4fbf82e390" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46547 } }
I20260810 18:52:31.099309 10134 raft_consensus.cc:388] T 00000000000000000000000000000000 P dca3eb6a54264cfe82820f4fbf82e390 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:52:31.099417 10134 raft_consensus.cc:482] T 00000000000000000000000000000000 P dca3eb6a54264cfe82820f4fbf82e390 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:52:31.099542 10134 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dca3eb6a54264cfe82820f4fbf82e390 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:31.100381 10137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:31.092422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37093 (local address 127.0.0.1:46547)
0810 18:52:31.092910 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:31.092924 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:31.092955 (+ 31us) server_negotiation.cc:407] Connection header received
0810 18:52:31.093814 (+ 859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:31.093841 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:31.094201 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:31.094515 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:31.094974 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:31.095992 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:31.097026 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:31.097522 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:31.097698 (+ 176us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:31.098585 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:31.098590 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:31.098602 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:52:31.099173 (+ 571us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:31.099227 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:52:31.099252 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:31.099420 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:31.099572 (+ 152us) server_negotiation.cc:299] Negotiation successful
0810 18:52:31.099707 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":160,"threads_started":1}
I20260810 18:52:31.100441 10134 raft_consensus.cc:504] T 00000000000000000000000000000000 P dca3eb6a54264cfe82820f4fbf82e390 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dca3eb6a54264cfe82820f4fbf82e390" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46547 } }
I20260810 18:52:31.100836 10134 leader_election.cc:302] T 00000000000000000000000000000000 P dca3eb6a54264cfe82820f4fbf82e390 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dca3eb6a54264cfe82820f4fbf82e390; no voters:
I20260810 18:52:31.101176 10134 leader_election.cc:288] T 00000000000000000000000000000000 P dca3eb6a54264cfe82820f4fbf82e390 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:52:31.101235 10139 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dca3eb6a54264cfe82820f4fbf82e390 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:52:31.101537 10139 raft_consensus.cc:686] T 00000000000000000000000000000000 P dca3eb6a54264cfe82820f4fbf82e390 [term 1 LEADER]: Becoming Leader. State: Replica: dca3eb6a54264cfe82820f4fbf82e390, State: Running, Role: LEADER
I20260810 18:52:31.101851 10139 consensus_queue.cc:237] T 00000000000000000000000000000000 P dca3eb6a54264cfe82820f4fbf82e390 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dca3eb6a54264cfe82820f4fbf82e390" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46547 } }
I20260810 18:52:31.102437 10134 sys_catalog.cc:564] T 00000000000000000000000000000000 P dca3eb6a54264cfe82820f4fbf82e390 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:52:31.103022 10139 sys_catalog.cc:455] T 00000000000000000000000000000000 P dca3eb6a54264cfe82820f4fbf82e390 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dca3eb6a54264cfe82820f4fbf82e390. Latest consensus state: current_term: 1 leader_uuid: "dca3eb6a54264cfe82820f4fbf82e390" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dca3eb6a54264cfe82820f4fbf82e390" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46547 } } }
I20260810 18:52:31.103142 10139 sys_catalog.cc:458] T 00000000000000000000000000000000 P dca3eb6a54264cfe82820f4fbf82e390 [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:31.103399 19106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:52:31.104460 10141 sys_catalog.cc:455] T 00000000000000000000000000000000 P dca3eb6a54264cfe82820f4fbf82e390 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dca3eb6a54264cfe82820f4fbf82e390" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dca3eb6a54264cfe82820f4fbf82e390" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46547 } } }
I20260810 18:52:31.104866 10141 sys_catalog.cc:458] T 00000000000000000000000000000000 P dca3eb6a54264cfe82820f4fbf82e390 [sys.catalog]: This master's current role is: LEADER
W20260810 18:52:31.107404 10157 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dca3eb6a54264cfe82820f4fbf82e390: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:52:31.107553 10157 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:52:31.107728 10147 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:52:31.108621 10147 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:52:31.110334 10137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:31.105893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60517 (local address 127.0.0.1:46547)
0810 18:52:31.106020 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:31.106024 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:31.106212 (+ 188us) server_negotiation.cc:407] Connection header received
0810 18:52:31.106328 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:31.106332 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:31.106375 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:31.106442 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:31.106822 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:31.107631 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:31.108557 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:31.108898 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:31.109016 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:31.109882 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:31.109886 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:31.109888 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:52:31.109900 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:31.109934 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:52:31.109946 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:31.110044 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:31.110167 (+ 123us) server_negotiation.cc:299] Negotiation successful
0810 18:52:31.110197 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 18:52:31.115557 10147 catalog_manager.cc:1347] Generated new cluster ID: c512a9b039cf407492d9352735565f61
I20260810 18:52:31.115664 10147 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:52:31.129536 10147 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:52:31.130455 10147 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:52:31.143594 10147 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dca3eb6a54264cfe82820f4fbf82e390: Generated new TSK 0
I20260810 18:52:31.144448 10147 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:52:31.156329 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 10055
W20260810 18:52:31.168183 10038 connection.cc:535] client connection to 127.0.0.1:46547 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46547): Connection reset by peer (error 104)
2026-08-10T18:52:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1516 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:52:31 dist-test-slave-cbsf krb5kdc[10171](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:52:31 dist-test-slave-cbsf krb5kdc[10171](info): set up 2 sockets
Aug 10 18:52:31 dist-test-slave-cbsf krb5kdc[10171](info): commencing operation
krb5kdc: starting...
W20260810 18:52:32.269181 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:52:32 dist-test-slave-cbsf krb5kdc[10171](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:52:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:52:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:52:32.391983 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44105
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44105
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:52:32.499821 10186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:32.500046 10186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:32.500084 10186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:32.503996 10186 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:52:32.504045 10186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:32.504065 10186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:32.504081 10186 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:52:32.504094 10186 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:52:32.509052 10186 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44105
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:32.509935 10186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:32.510821 10186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:32.518985 10194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:32.518965 10192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:32.519107 10186 server_base.cc:1034] running on GCE node
W20260810 18:52:32.518965 10191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:32.519770 10186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:32.520818 10186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:32.521972 10186 hybrid_clock.cc:648] HybridClock initialized: now 1786387952521959 us; error 29 us; skew 500 ppm
Aug 10 18:52:32 dist-test-slave-cbsf krb5kdc[10171](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387952, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:32.528882 10186 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:52:32.529760 10186 webserver.cc:460] Webserver started at http://127.0.0.1:38705/ using document root <none> and password file <none>
I20260810 18:52:32.530269 10186 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:32.530354 10186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:32.530557 10186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:32.532676 10186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "f9ae5085c05e4bd58a3390d2b58122e6"
format_stamp: "Formatted at 2026-08-10 18:52:32 on dist-test-slave-cbsf"
server_key: "e39d6f16f139622cfce48653ac4e11bb"
server_key_iv: "7d546a687baf5078a5a0b02d96486113"
server_key_version: "encryptionkey@0"
I20260810 18:52:32.533294 10186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "f9ae5085c05e4bd58a3390d2b58122e6"
format_stamp: "Formatted at 2026-08-10 18:52:32 on dist-test-slave-cbsf"
server_key: "e39d6f16f139622cfce48653ac4e11bb"
server_key_iv: "7d546a687baf5078a5a0b02d96486113"
server_key_version: "encryptionkey@0"
I20260810 18:52:32.536814 10186 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260810 18:52:32.539317 10203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:32.540416 10186 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:52:32.540550 10186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "f9ae5085c05e4bd58a3390d2b58122e6"
format_stamp: "Formatted at 2026-08-10 18:52:32 on dist-test-slave-cbsf"
server_key: "e39d6f16f139622cfce48653ac4e11bb"
server_key_iv: "7d546a687baf5078a5a0b02d96486113"
server_key_version: "encryptionkey@0"
I20260810 18:52:32.540647 10186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:32.545125 10186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:32.547706 10186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:32.547891 10186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:32.555334 10186 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44105
I20260810 18:52:32.555362 10263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44105 every 8 connection(s)
I20260810 18:52:32.556195 10186 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:52:32.557726 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 10186
I20260810 18:52:32.557891 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:52:32.558218 19106 external_mini_cluster.cc:1428] Setting key c9b7453cdb134806d6ceac7986643b91
I20260810 18:52:32.559230 10264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:32.563660 10270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:32.559577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47893 (local address 127.0.0.1:44105)
0810 18:52:32.559985 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:32.560013 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:32.560076 (+ 63us) server_negotiation.cc:407] Connection header received
0810 18:52:32.560936 (+ 860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:32.560959 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:32.561288 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:32.561643 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:32.562338 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:32.562344 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:32.562360 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:52:32.562373 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:32.562435 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:52:32.562461 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:32.562624 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:32.562690 (+ 66us) server_negotiation.cc:299] Negotiation successful
0810 18:52:32.562827 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":105,"threads_started":1}
I20260810 18:52:32.566074 10264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9ae5085c05e4bd58a3390d2b58122e6: Bootstrap starting.
I20260810 18:52:32.566726 19106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:52:32.568317 10264 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9ae5085c05e4bd58a3390d2b58122e6: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:32.569121 10264 log.cc:824] T 00000000000000000000000000000000 P f9ae5085c05e4bd58a3390d2b58122e6: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:32.570578 10270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:32.569047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55527 (local address 127.0.0.1:44105)
0810 18:52:32.569178 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:32.569182 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:32.569402 (+ 220us) server_negotiation.cc:407] Connection header received
0810 18:52:32.569533 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:32.569537 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:32.569573 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:32.569658 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:32.570277 (+ 619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:32.570279 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:32.570281 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:52:32.570291 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:32.570315 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:52:32.570326 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:32.570395 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:32.570475 (+ 80us) server_negotiation.cc:299] Negotiation successful
0810 18:52:32.570502 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 18:52:32.571276 10264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9ae5085c05e4bd58a3390d2b58122e6: No bootstrap required, opened a new log
I20260810 18:52:32.575071 10264 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9ae5085c05e4bd58a3390d2b58122e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9ae5085c05e4bd58a3390d2b58122e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44105 } }
I20260810 18:52:32.575268 10264 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9ae5085c05e4bd58a3390d2b58122e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:32.575302 10264 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9ae5085c05e4bd58a3390d2b58122e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9ae5085c05e4bd58a3390d2b58122e6, State: Initialized, Role: FOLLOWER
I20260810 18:52:32.575675 10264 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9ae5085c05e4bd58a3390d2b58122e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9ae5085c05e4bd58a3390d2b58122e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44105 } }
I20260810 18:52:32.575839 10264 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9ae5085c05e4bd58a3390d2b58122e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:52:32.575891 10264 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9ae5085c05e4bd58a3390d2b58122e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:52:32.575953 10264 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9ae5085c05e4bd58a3390d2b58122e6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:32.576743 10264 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9ae5085c05e4bd58a3390d2b58122e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9ae5085c05e4bd58a3390d2b58122e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44105 } }
I20260810 18:52:32.577105 10264 leader_election.cc:302] T 00000000000000000000000000000000 P f9ae5085c05e4bd58a3390d2b58122e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9ae5085c05e4bd58a3390d2b58122e6; no voters:
I20260810 18:52:32.577513 10264 leader_election.cc:288] T 00000000000000000000000000000000 P f9ae5085c05e4bd58a3390d2b58122e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:52:32.577637 10280 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9ae5085c05e4bd58a3390d2b58122e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:52:32.577890 10280 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9ae5085c05e4bd58a3390d2b58122e6 [term 1 LEADER]: Becoming Leader. State: Replica: f9ae5085c05e4bd58a3390d2b58122e6, State: Running, Role: LEADER
I20260810 18:52:32.578222 10280 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9ae5085c05e4bd58a3390d2b58122e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9ae5085c05e4bd58a3390d2b58122e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44105 } }
I20260810 18:52:32.578670 10264 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9ae5085c05e4bd58a3390d2b58122e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:52:32.579852 10282 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9ae5085c05e4bd58a3390d2b58122e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9ae5085c05e4bd58a3390d2b58122e6. Latest consensus state: current_term: 1 leader_uuid: "f9ae5085c05e4bd58a3390d2b58122e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9ae5085c05e4bd58a3390d2b58122e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44105 } } }
I20260810 18:52:32.579882 10281 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9ae5085c05e4bd58a3390d2b58122e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f9ae5085c05e4bd58a3390d2b58122e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9ae5085c05e4bd58a3390d2b58122e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44105 } } }
I20260810 18:52:32.580021 10281 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9ae5085c05e4bd58a3390d2b58122e6 [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:32.580287 10282 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9ae5085c05e4bd58a3390d2b58122e6 [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:32.580456 10290 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:52:32.583127 10290 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:52:32.588116 10290 catalog_manager.cc:1347] Generated new cluster ID: 027f83c6a58c4c9f8f388074ad8fa2c0
I20260810 18:52:32.588171 10290 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:52:32.610899 10290 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:52:32.611215 10290 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:52:32.616760 10290 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f9ae5085c05e4bd58a3390d2b58122e6: Generated new TSK 0
I20260810 18:52:32.617467 10290 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:52:32.627681 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 10186
2026-08-10T18:52:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1468 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:52:32 dist-test-slave-cbsf krb5kdc[10303](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:52:32 dist-test-slave-cbsf krb5kdc[10303](info): set up 2 sockets
Aug 10 18:52:32 dist-test-slave-cbsf krb5kdc[10303](info): commencing operation
krb5kdc: starting...
W20260810 18:52:33.722754 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:52:33 dist-test-slave-cbsf krb5kdc[10303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:52:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:52:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:52:33.849670 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42523
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42523
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:52:33.960400 10317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:33.960678 10317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:33.960806 10317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:33.964745 10317 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:52:33.964797 10317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:33.964815 10317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:33.964833 10317 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:52:33.964876 10317 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:52:33.971453 10317 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42523
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:33.972465 10317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:33.973536 10317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:33.982563 10326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:33.982570 10323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:33.982643 10324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:33.982923 10317 server_base.cc:1034] running on GCE node
I20260810 18:52:33.983583 10317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:33.984898 10317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:33.986125 10317 hybrid_clock.cc:648] HybridClock initialized: now 1786387953986108 us; error 36 us; skew 500 ppm
Aug 10 18:52:33 dist-test-slave-cbsf krb5kdc[10303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387953, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:33.989094 10317 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:52:33.990137 10317 webserver.cc:460] Webserver started at http://127.0.0.1:41427/ using document root <none> and password file <none>
I20260810 18:52:33.990811 10317 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:33.990921 10317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:33.991194 10317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:33.993592 10317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "d2f764d1bdf34694a70fb8e6b97dea70"
format_stamp: "Formatted at 2026-08-10 18:52:33 on dist-test-slave-cbsf"
server_key: "4a2ca1721ac1427ff65b5a342807db5a"
server_key_iv: "cbb260780f10aa5b2671ee8d63ff581d"
server_key_version: "encryptionkey@0"
I20260810 18:52:33.994297 10317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "d2f764d1bdf34694a70fb8e6b97dea70"
format_stamp: "Formatted at 2026-08-10 18:52:33 on dist-test-slave-cbsf"
server_key: "4a2ca1721ac1427ff65b5a342807db5a"
server_key_iv: "cbb260780f10aa5b2671ee8d63ff581d"
server_key_version: "encryptionkey@0"
I20260810 18:52:33.998354 10317 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:52:34.001226 10333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:34.002856 10317 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:52:34.003063 10317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "d2f764d1bdf34694a70fb8e6b97dea70"
format_stamp: "Formatted at 2026-08-10 18:52:33 on dist-test-slave-cbsf"
server_key: "4a2ca1721ac1427ff65b5a342807db5a"
server_key_iv: "cbb260780f10aa5b2671ee8d63ff581d"
server_key_version: "encryptionkey@0"
I20260810 18:52:34.003254 10317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:34.033610 10317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:34.036808 10317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:34.037015 10317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:34.044282 10317 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42523
I20260810 18:52:34.044296 10393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42523 every 8 connection(s)
I20260810 18:52:34.045286 10317 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:52:34.046558 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 10317
I20260810 18:52:34.046707 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:52:34.047046 19106 external_mini_cluster.cc:1428] Setting key 60068b5830eb6855dc71701e022df170
I20260810 18:52:34.048609 10394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:34.055181 10394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2f764d1bdf34694a70fb8e6b97dea70: Bootstrap starting.
Aug 10 18:52:34 dist-test-slave-cbsf krb5kdc[10303](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 18:52:34.057868 10394 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2f764d1bdf34694a70fb8e6b97dea70: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:34.058597 10394 log.cc:824] T 00000000000000000000000000000000 P d2f764d1bdf34694a70fb8e6b97dea70: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:34.060825 10394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2f764d1bdf34694a70fb8e6b97dea70: No bootstrap required, opened a new log
I20260810 18:52:34.062749 10400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:34.048512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40985 (local address 127.0.0.1:42523)
0810 18:52:34.048996 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:34.049029 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:34.049066 (+ 37us) server_negotiation.cc:407] Connection header received
0810 18:52:34.049771 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:34.049789 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:34.050135 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:34.050437 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:34.051360 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:34.052612 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:34.053822 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:34.054334 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:34.054479 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:34.057425 (+ 2946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:34.057430 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:34.057442 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:34.057504 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:34.059530 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:34.060015 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:34.060019 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:34.060024 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:34.060130 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:34.060440 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:34.060443 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:34.060444 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:34.061012 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:34.061171 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:34.061464 (+ 293us) server_negotiation.cc:299] Negotiation successful
0810 18:52:34.061808 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":191,"threads_started":1}
I20260810 18:52:34.064226 10394 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2f764d1bdf34694a70fb8e6b97dea70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2f764d1bdf34694a70fb8e6b97dea70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42523 } }
I20260810 18:52:34.064512 10394 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2f764d1bdf34694a70fb8e6b97dea70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:34.064611 10394 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2f764d1bdf34694a70fb8e6b97dea70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2f764d1bdf34694a70fb8e6b97dea70, State: Initialized, Role: FOLLOWER
I20260810 18:52:34.065160 10394 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2f764d1bdf34694a70fb8e6b97dea70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2f764d1bdf34694a70fb8e6b97dea70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42523 } }
I20260810 18:52:34.065328 10394 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2f764d1bdf34694a70fb8e6b97dea70 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:52:34.065393 10394 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2f764d1bdf34694a70fb8e6b97dea70 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:52:34.065498 10394 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2f764d1bdf34694a70fb8e6b97dea70 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:34.065693 19106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:52:34.066504 10394 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2f764d1bdf34694a70fb8e6b97dea70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2f764d1bdf34694a70fb8e6b97dea70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42523 } }
I20260810 18:52:34.066847 10394 leader_election.cc:302] T 00000000000000000000000000000000 P d2f764d1bdf34694a70fb8e6b97dea70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2f764d1bdf34694a70fb8e6b97dea70; no voters:
I20260810 18:52:34.067236 10394 leader_election.cc:288] T 00000000000000000000000000000000 P d2f764d1bdf34694a70fb8e6b97dea70 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:52:34.067317 10408 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2f764d1bdf34694a70fb8e6b97dea70 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:52:34.067613 10408 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2f764d1bdf34694a70fb8e6b97dea70 [term 1 LEADER]: Becoming Leader. State: Replica: d2f764d1bdf34694a70fb8e6b97dea70, State: Running, Role: LEADER
I20260810 18:52:34.067950 10408 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2f764d1bdf34694a70fb8e6b97dea70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2f764d1bdf34694a70fb8e6b97dea70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42523 } }
I20260810 18:52:34.068760 10394 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2f764d1bdf34694a70fb8e6b97dea70 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:52:34.069386 10410 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2f764d1bdf34694a70fb8e6b97dea70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2f764d1bdf34694a70fb8e6b97dea70. Latest consensus state: current_term: 1 leader_uuid: "d2f764d1bdf34694a70fb8e6b97dea70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2f764d1bdf34694a70fb8e6b97dea70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42523 } } }
I20260810 18:52:34.070235 10410 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2f764d1bdf34694a70fb8e6b97dea70 [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:34.070101 10409 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2f764d1bdf34694a70fb8e6b97dea70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d2f764d1bdf34694a70fb8e6b97dea70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2f764d1bdf34694a70fb8e6b97dea70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42523 } } }
I20260810 18:52:34.070568 10409 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2f764d1bdf34694a70fb8e6b97dea70 [sys.catalog]: This master's current role is: LEADER
W20260810 18:52:34.074486 10429 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d2f764d1bdf34694a70fb8e6b97dea70: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:52:34.074581 10429 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:52:34.074648 10423 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:52:34.075377 10423 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:52:34.077411 10400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:34.068151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49719 (local address 127.0.0.1:42523)
0810 18:52:34.068248 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:34.068252 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:34.068438 (+ 186us) server_negotiation.cc:407] Connection header received
0810 18:52:34.068585 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:34.068591 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:34.068643 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:34.068753 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:34.069712 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:34.070580 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:34.071744 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:34.071972 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:34.072059 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:34.073698 (+ 1639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:34.073701 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:34.073704 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:34.073730 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:34.075469 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:34.075959 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:34.075962 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:34.075964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:34.076058 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:34.076381 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:34.076385 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:34.076387 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:34.076637 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:34.076810 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:34.077090 (+ 280us) server_negotiation.cc:299] Negotiation successful
0810 18:52:34.077239 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 18:52:34.081655 10423 catalog_manager.cc:1347] Generated new cluster ID: 76ab4e22ac254d1d8c19b00c0fc27e88
I20260810 18:52:34.081724 10423 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:52:34.100210 10423 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:52:34.101284 10423 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:52:34.107703 10423 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2f764d1bdf34694a70fb8e6b97dea70: Generated new TSK 0
I20260810 18:52:34.108453 10423 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:52:34.120085 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 10317
2026-08-10T18:52:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1491 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:52:34 dist-test-slave-cbsf krb5kdc[10437](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:52:34 dist-test-slave-cbsf krb5kdc[10437](info): set up 2 sockets
Aug 10 18:52:34 dist-test-slave-cbsf krb5kdc[10437](info): commencing operation
krb5kdc: starting...
W20260810 18:52:35.249562 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:52:35 dist-test-slave-cbsf krb5kdc[10437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387955, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:52:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:52:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:52:35.375599 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45611
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45611
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:52:35.489344 10451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:35.489581 10451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:35.489619 10451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:35.493538 10451 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:52:35.493589 10451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:35.493607 10451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:35.493655 10451 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:52:35.493686 10451 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:52:35.498868 10451 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45611
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:35.499912 10451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:35.500952 10451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:35.509124 10457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:35.509155 10460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:35.509124 10458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:35.509373 10451 server_base.cc:1034] running on GCE node
I20260810 18:52:35.510155 10451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:35.511590 10451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:35.512768 10451 hybrid_clock.cc:648] HybridClock initialized: now 1786387955512748 us; error 33 us; skew 500 ppm
Aug 10 18:52:35 dist-test-slave-cbsf krb5kdc[10437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387955, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:35.516470 10451 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:52:35.517885 10451 webserver.cc:460] Webserver started at http://127.0.0.1:39767/ using document root <none> and password file <none>
I20260810 18:52:35.518734 10451 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:35.518828 10451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:35.519080 10451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:35.521685 10451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "3ecbdbb70d2246528c2ce7323446fa60"
format_stamp: "Formatted at 2026-08-10 18:52:35 on dist-test-slave-cbsf"
server_key: "c07483714c70617b4405202f3ba19af2"
server_key_iv: "3bb27c968edba93431f24649fc819a60"
server_key_version: "encryptionkey@0"
I20260810 18:52:35.522195 10451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "3ecbdbb70d2246528c2ce7323446fa60"
format_stamp: "Formatted at 2026-08-10 18:52:35 on dist-test-slave-cbsf"
server_key: "c07483714c70617b4405202f3ba19af2"
server_key_iv: "3bb27c968edba93431f24649fc819a60"
server_key_version: "encryptionkey@0"
I20260810 18:52:35.525688 10451 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:52:35.527998 10467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:35.529337 10451 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:52:35.529438 10451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "3ecbdbb70d2246528c2ce7323446fa60"
format_stamp: "Formatted at 2026-08-10 18:52:35 on dist-test-slave-cbsf"
server_key: "c07483714c70617b4405202f3ba19af2"
server_key_iv: "3bb27c968edba93431f24649fc819a60"
server_key_version: "encryptionkey@0"
I20260810 18:52:35.529523 10451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:35.552271 10451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:35.555090 10451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:35.555241 10451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:35.562438 10451 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45611
I20260810 18:52:35.562465 10529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45611 every 8 connection(s)
I20260810 18:52:35.563370 10451 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:52:35.566125 10530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:35.571403 10530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ecbdbb70d2246528c2ce7323446fa60: Bootstrap starting.
I20260810 18:52:35.572636 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 10451
I20260810 18:52:35.572798 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:52:35.573124 19106 external_mini_cluster.cc:1428] Setting key ea5ea95b665a4b516e2f0a05118bb0d8
I20260810 18:52:35.574224 10530 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ecbdbb70d2246528c2ce7323446fa60: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:35.575063 10530 log.cc:824] T 00000000000000000000000000000000 P 3ecbdbb70d2246528c2ce7323446fa60: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:35.577001 10530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ecbdbb70d2246528c2ce7323446fa60: No bootstrap required, opened a new log
I20260810 18:52:35.580458 10530 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ecbdbb70d2246528c2ce7323446fa60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ecbdbb70d2246528c2ce7323446fa60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45611 } }
I20260810 18:52:35.580729 10530 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ecbdbb70d2246528c2ce7323446fa60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:35.580814 10530 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ecbdbb70d2246528c2ce7323446fa60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ecbdbb70d2246528c2ce7323446fa60, State: Initialized, Role: FOLLOWER
I20260810 18:52:35.581310 10530 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ecbdbb70d2246528c2ce7323446fa60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ecbdbb70d2246528c2ce7323446fa60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45611 } }
I20260810 18:52:35.581436 10530 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ecbdbb70d2246528c2ce7323446fa60 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:52:35.581521 10530 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ecbdbb70d2246528c2ce7323446fa60 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:52:35.581663 10530 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ecbdbb70d2246528c2ce7323446fa60 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:35.582201 10533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:35.574255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58101 (local address 127.0.0.1:45611)
0810 18:52:35.574724 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:35.574739 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:35.574772 (+ 33us) server_negotiation.cc:407] Connection header received
0810 18:52:35.575524 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:35.575547 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:35.575943 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:35.576267 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:35.576749 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:35.577795 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:35.578719 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:35.579209 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:35.579382 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:35.580234 (+ 852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:35.580239 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:35.580252 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:52:35.580643 (+ 391us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:35.580751 (+ 108us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:52:35.580788 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:35.580928 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:35.581054 (+ 126us) server_negotiation.cc:299] Negotiation successful
0810 18:52:35.581292 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":145,"threads_started":1}
I20260810 18:52:35.582594 10530 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ecbdbb70d2246528c2ce7323446fa60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ecbdbb70d2246528c2ce7323446fa60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45611 } }
I20260810 18:52:35.582940 10530 leader_election.cc:302] T 00000000000000000000000000000000 P 3ecbdbb70d2246528c2ce7323446fa60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ecbdbb70d2246528c2ce7323446fa60; no voters:
I20260810 18:52:35.583254 10530 leader_election.cc:288] T 00000000000000000000000000000000 P 3ecbdbb70d2246528c2ce7323446fa60 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:52:35.583356 10535 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ecbdbb70d2246528c2ce7323446fa60 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:52:35.583603 10535 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ecbdbb70d2246528c2ce7323446fa60 [term 1 LEADER]: Becoming Leader. State: Replica: 3ecbdbb70d2246528c2ce7323446fa60, State: Running, Role: LEADER
I20260810 18:52:35.583937 10535 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ecbdbb70d2246528c2ce7323446fa60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ecbdbb70d2246528c2ce7323446fa60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45611 } }
I20260810 18:52:35.584455 10530 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ecbdbb70d2246528c2ce7323446fa60 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:52:35.585764 10539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ecbdbb70d2246528c2ce7323446fa60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ecbdbb70d2246528c2ce7323446fa60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ecbdbb70d2246528c2ce7323446fa60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45611 } } }
I20260810 18:52:35.585817 10537 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ecbdbb70d2246528c2ce7323446fa60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ecbdbb70d2246528c2ce7323446fa60. Latest consensus state: current_term: 1 leader_uuid: "3ecbdbb70d2246528c2ce7323446fa60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ecbdbb70d2246528c2ce7323446fa60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45611 } } }
I20260810 18:52:35.585882 10539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ecbdbb70d2246528c2ce7323446fa60 [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:35.585911 10537 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ecbdbb70d2246528c2ce7323446fa60 [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:35.586206 19106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:52:35.586264 10546 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:52:35.589299 10546 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:52:35.595144 10546 catalog_manager.cc:1347] Generated new cluster ID: 99083e88fb59412a82f27e78d59c6c98
I20260810 18:52:35.595230 10546 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:52:35.597675 10533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:35.588390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59711 (local address 127.0.0.1:45611)
0810 18:52:35.588508 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:35.588513 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:35.591390 (+ 2877us) server_negotiation.cc:407] Connection header received
0810 18:52:35.591504 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:35.591506 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:35.591552 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:35.591632 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:35.592051 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:35.592868 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:35.594003 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:35.594326 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:35.594433 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:35.597184 (+ 2751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:35.597188 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:35.597190 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:52:35.597222 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:35.597253 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:52:35.597273 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:35.597314 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:35.597474 (+ 160us) server_negotiation.cc:299] Negotiation successful
0810 18:52:35.597527 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260810 18:52:35.604821 10546 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:52:35.605947 10546 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:52:35.620868 10546 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ecbdbb70d2246528c2ce7323446fa60: Generated new TSK 0
I20260810 18:52:35.621712 10546 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:52:35.630412 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 10451
2026-08-10T18:52:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1512 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:52:35 dist-test-slave-cbsf krb5kdc[10566](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:52:35 dist-test-slave-cbsf krb5kdc[10566](info): set up 2 sockets
Aug 10 18:52:35 dist-test-slave-cbsf krb5kdc[10566](info): commencing operation
krb5kdc: starting...
W20260810 18:52:36.782271 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:52:36 dist-test-slave-cbsf krb5kdc[10566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387956, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:52:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:52:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:52:36.907967 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37161
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37161
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:52:37.026057 10579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:37.026372 10579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:37.026460 10579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:37.030750 10579 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:52:37.030841 10579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:37.030879 10579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:37.030943 10579 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:52:37.030980 10579 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:52:37.036340 10579 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37161
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:37.037492 10579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:37.038529 10579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:37.047245 10591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:37.047253 10588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:37.047400 10585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:37.047646 10579 server_base.cc:1034] running on GCE node
I20260810 18:52:37.048171 10579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:37.049305 10579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:37.050499 10579 hybrid_clock.cc:648] HybridClock initialized: now 1786387957050479 us; error 30 us; skew 500 ppm
Aug 10 18:52:37 dist-test-slave-cbsf krb5kdc[10566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387957, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:37.053680 10579 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:52:37.054615 10579 webserver.cc:460] Webserver started at http://127.0.0.1:45211/ using document root <none> and password file <none>
I20260810 18:52:37.055193 10579 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:37.055286 10579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:37.055531 10579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:37.057650 10579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "caa523b218e84a6e8c312cab747e6ea7"
format_stamp: "Formatted at 2026-08-10 18:52:37 on dist-test-slave-cbsf"
server_key: "e2d04bc7c689b60529eebea3e76f13c0"
server_key_iv: "2b0139453fc22e05303df0a40319dd65"
server_key_version: "encryptionkey@0"
I20260810 18:52:37.058235 10579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "caa523b218e84a6e8c312cab747e6ea7"
format_stamp: "Formatted at 2026-08-10 18:52:37 on dist-test-slave-cbsf"
server_key: "e2d04bc7c689b60529eebea3e76f13c0"
server_key_iv: "2b0139453fc22e05303df0a40319dd65"
server_key_version: "encryptionkey@0"
I20260810 18:52:37.061882 10579 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:52:37.064298 10598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:37.065573 10579 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:52:37.065714 10579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "caa523b218e84a6e8c312cab747e6ea7"
format_stamp: "Formatted at 2026-08-10 18:52:37 on dist-test-slave-cbsf"
server_key: "e2d04bc7c689b60529eebea3e76f13c0"
server_key_iv: "2b0139453fc22e05303df0a40319dd65"
server_key_version: "encryptionkey@0"
I20260810 18:52:37.065821 10579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:37.070425 10579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:37.073313 10579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:37.073506 10579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:37.080440 10579 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37161
I20260810 18:52:37.080596 10654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37161 every 8 connection(s)
I20260810 18:52:37.081420 10579 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:52:37.084201 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 10579
I20260810 18:52:37.084371 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:52:37.084620 10655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:37.084735 19106 external_mini_cluster.cc:1428] Setting key c8fa61edeca39c2f03c49489cd4539ea
I20260810 18:52:37.090274 10659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:37.086157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39939 (local address 127.0.0.1:37161)
0810 18:52:37.086575 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:37.086595 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:37.086635 (+ 40us) server_negotiation.cc:407] Connection header received
0810 18:52:37.087476 (+ 841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:37.087501 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:37.087877 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:37.088244 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:37.088936 (+ 692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:37.088941 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:37.088957 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:52:37.088968 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:37.089029 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:52:37.089054 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:37.089188 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:37.089251 (+ 63us) server_negotiation.cc:299] Negotiation successful
0810 18:52:37.089395 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":103,"threads_started":1}
I20260810 18:52:37.092176 10655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P caa523b218e84a6e8c312cab747e6ea7: Bootstrap starting.
I20260810 18:52:37.093209 19106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:52:37.094659 10655 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P caa523b218e84a6e8c312cab747e6ea7: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:37.095561 10655 log.cc:824] T 00000000000000000000000000000000 P caa523b218e84a6e8c312cab747e6ea7: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:37.097093 10659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:37.095250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41153 (local address 127.0.0.1:37161)
0810 18:52:37.095361 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:37.095366 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:37.095568 (+ 202us) server_negotiation.cc:407] Connection header received
0810 18:52:37.095705 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:37.095708 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:37.095743 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:37.095826 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:37.096631 (+ 805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:37.096635 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:37.096637 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:52:37.096648 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:37.096674 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:52:37.096687 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:37.096765 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:37.096949 (+ 184us) server_negotiation.cc:299] Negotiation successful
0810 18:52:37.096989 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 18:52:37.097839 10655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P caa523b218e84a6e8c312cab747e6ea7: No bootstrap required, opened a new log
I20260810 18:52:37.101578 10655 raft_consensus.cc:348] T 00000000000000000000000000000000 P caa523b218e84a6e8c312cab747e6ea7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa523b218e84a6e8c312cab747e6ea7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37161 } }
I20260810 18:52:37.101778 10655 raft_consensus.cc:374] T 00000000000000000000000000000000 P caa523b218e84a6e8c312cab747e6ea7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:37.101877 10655 raft_consensus.cc:729] T 00000000000000000000000000000000 P caa523b218e84a6e8c312cab747e6ea7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caa523b218e84a6e8c312cab747e6ea7, State: Initialized, Role: FOLLOWER
I20260810 18:52:37.102291 10655 consensus_queue.cc:260] T 00000000000000000000000000000000 P caa523b218e84a6e8c312cab747e6ea7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa523b218e84a6e8c312cab747e6ea7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37161 } }
I20260810 18:52:37.102439 10655 raft_consensus.cc:388] T 00000000000000000000000000000000 P caa523b218e84a6e8c312cab747e6ea7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:52:37.102507 10655 raft_consensus.cc:482] T 00000000000000000000000000000000 P caa523b218e84a6e8c312cab747e6ea7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:52:37.102628 10655 raft_consensus.cc:3037] T 00000000000000000000000000000000 P caa523b218e84a6e8c312cab747e6ea7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:37.103462 10655 raft_consensus.cc:504] T 00000000000000000000000000000000 P caa523b218e84a6e8c312cab747e6ea7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa523b218e84a6e8c312cab747e6ea7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37161 } }
I20260810 18:52:37.103830 10655 leader_election.cc:302] T 00000000000000000000000000000000 P caa523b218e84a6e8c312cab747e6ea7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: caa523b218e84a6e8c312cab747e6ea7; no voters:
I20260810 18:52:37.104112 10655 leader_election.cc:288] T 00000000000000000000000000000000 P caa523b218e84a6e8c312cab747e6ea7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:52:37.104231 10669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P caa523b218e84a6e8c312cab747e6ea7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:52:37.104521 10669 raft_consensus.cc:686] T 00000000000000000000000000000000 P caa523b218e84a6e8c312cab747e6ea7 [term 1 LEADER]: Becoming Leader. State: Replica: caa523b218e84a6e8c312cab747e6ea7, State: Running, Role: LEADER
I20260810 18:52:37.104941 10669 consensus_queue.cc:237] T 00000000000000000000000000000000 P caa523b218e84a6e8c312cab747e6ea7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa523b218e84a6e8c312cab747e6ea7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37161 } }
I20260810 18:52:37.106539 10655 sys_catalog.cc:564] T 00000000000000000000000000000000 P caa523b218e84a6e8c312cab747e6ea7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:52:37.106822 10670 sys_catalog.cc:455] T 00000000000000000000000000000000 P caa523b218e84a6e8c312cab747e6ea7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "caa523b218e84a6e8c312cab747e6ea7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa523b218e84a6e8c312cab747e6ea7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37161 } } }
I20260810 18:52:37.106894 10671 sys_catalog.cc:455] T 00000000000000000000000000000000 P caa523b218e84a6e8c312cab747e6ea7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader caa523b218e84a6e8c312cab747e6ea7. Latest consensus state: current_term: 1 leader_uuid: "caa523b218e84a6e8c312cab747e6ea7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa523b218e84a6e8c312cab747e6ea7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37161 } } }
I20260810 18:52:37.106954 10670 sys_catalog.cc:458] T 00000000000000000000000000000000 P caa523b218e84a6e8c312cab747e6ea7 [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:37.107043 10671 sys_catalog.cc:458] T 00000000000000000000000000000000 P caa523b218e84a6e8c312cab747e6ea7 [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:37.107398 10673 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:52:37.110435 10673 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:52:37.115427 10673 catalog_manager.cc:1347] Generated new cluster ID: d6c5b43d954844ebbd1bd8669bb101c9
I20260810 18:52:37.115492 10673 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:52:37.131991 10673 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:52:37.132310 10673 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:52:37.141593 10673 catalog_manager.cc:5868] T 00000000000000000000000000000000 P caa523b218e84a6e8c312cab747e6ea7: Generated new TSK 0
I20260810 18:52:37.142227 10673 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:52:37.153508 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 10579
2026-08-10T18:52:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1518 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:52:37 dist-test-slave-cbsf krb5kdc[10692](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:52:37 dist-test-slave-cbsf krb5kdc[10692](info): set up 2 sockets
Aug 10 18:52:37 dist-test-slave-cbsf krb5kdc[10692](info): commencing operation
krb5kdc: starting...
W20260810 18:52:38.260468 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:52:38 dist-test-slave-cbsf krb5kdc[10692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387958, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:52:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:52:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:52:38.388007 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46693
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46693
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:52:38.497185 10707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:38.497467 10707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:38.497535 10707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:38.501549 10707 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:52:38.501627 10707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:38.501667 10707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:38.501703 10707 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:52:38.501735 10707 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:52:38.506733 10707 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46693
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10707
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:38.507737 10707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:38.508759 10707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:38.516908 10713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:38.516920 10712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:38.516940 10715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:38.517202 10707 server_base.cc:1034] running on GCE node
I20260810 18:52:38.517794 10707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:38.519196 10707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:38.520385 10707 hybrid_clock.cc:648] HybridClock initialized: now 1786387958520373 us; error 34 us; skew 500 ppm
Aug 10 18:52:38 dist-test-slave-cbsf krb5kdc[10692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387958, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:38.527504 10707 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:52:38.528391 10707 webserver.cc:460] Webserver started at http://127.0.0.1:38141/ using document root <none> and password file <none>
I20260810 18:52:38.529043 10707 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:38.529151 10707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:38.529362 10707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:38.531309 10707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "1d95bd60277849b18746b22a96850237"
format_stamp: "Formatted at 2026-08-10 18:52:38 on dist-test-slave-cbsf"
server_key: "447e034efc6c2100d98f97eb3646f012"
server_key_iv: "94083a0b1b517ad8f5a779dbb6113c79"
server_key_version: "encryptionkey@0"
I20260810 18:52:38.531864 10707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "1d95bd60277849b18746b22a96850237"
format_stamp: "Formatted at 2026-08-10 18:52:38 on dist-test-slave-cbsf"
server_key: "447e034efc6c2100d98f97eb3646f012"
server_key_iv: "94083a0b1b517ad8f5a779dbb6113c79"
server_key_version: "encryptionkey@0"
I20260810 18:52:38.535542 10707 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260810 18:52:38.538055 10724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:38.539186 10707 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:52:38.539321 10707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "1d95bd60277849b18746b22a96850237"
format_stamp: "Formatted at 2026-08-10 18:52:38 on dist-test-slave-cbsf"
server_key: "447e034efc6c2100d98f97eb3646f012"
server_key_iv: "94083a0b1b517ad8f5a779dbb6113c79"
server_key_version: "encryptionkey@0"
I20260810 18:52:38.539423 10707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:38.552057 10707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:38.554898 10707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:38.555044 10707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:38.562017 10707 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46693
I20260810 18:52:38.562107 10781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46693 every 8 connection(s)
I20260810 18:52:38.562851 10707 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:52:38.563992 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 10707
I20260810 18:52:38.564116 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:52:38.564332 19106 external_mini_cluster.cc:1428] Setting key 6e542964d6460b2af3a5bdc11c6cda38
I20260810 18:52:38.565717 10782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:38.572250 10782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d95bd60277849b18746b22a96850237: Bootstrap starting.
Aug 10 18:52:38 dist-test-slave-cbsf krb5kdc[10692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387958, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 18:52:38.574723 10782 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d95bd60277849b18746b22a96850237: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:38.575659 10782 log.cc:824] T 00000000000000000000000000000000 P 1d95bd60277849b18746b22a96850237: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:38.577903 10782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d95bd60277849b18746b22a96850237: No bootstrap required, opened a new log
I20260810 18:52:38.579133 10785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:38.565611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.129:33125 (local address 127.0.0.1:46693)
0810 18:52:38.566006 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:38.566024 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:38.566062 (+ 38us) server_negotiation.cc:407] Connection header received
0810 18:52:38.566877 (+ 815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:38.566899 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:38.567311 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:38.567649 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:38.568340 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:38.569834 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:38.570650 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:38.571381 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:38.571533 (+ 152us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:38.574055 (+ 2522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:38.574061 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:38.574075 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:38.574105 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:38.576080 (+ 1975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:38.576645 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:38.576650 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:38.576655 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:38.576803 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:38.577125 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:38.577129 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:38.577134 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:38.577625 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:38.577842 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:38.578056 (+ 214us) server_negotiation.cc:299] Negotiation successful
0810 18:52:38.578291 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":141,"threads_started":1}
I20260810 18:52:38.582150 19106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:52:38.582084 10782 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d95bd60277849b18746b22a96850237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d95bd60277849b18746b22a96850237" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46693 } }
I20260810 18:52:38.582326 10782 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d95bd60277849b18746b22a96850237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:38.582386 10782 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d95bd60277849b18746b22a96850237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d95bd60277849b18746b22a96850237, State: Initialized, Role: FOLLOWER
I20260810 18:52:38.582760 10782 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d95bd60277849b18746b22a96850237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d95bd60277849b18746b22a96850237" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46693 } }
I20260810 18:52:38.582873 10782 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d95bd60277849b18746b22a96850237 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:52:38.582959 10782 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d95bd60277849b18746b22a96850237 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:52:38.583069 10782 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d95bd60277849b18746b22a96850237 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:38.584059 10782 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d95bd60277849b18746b22a96850237 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d95bd60277849b18746b22a96850237" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46693 } }
I20260810 18:52:38.584409 10782 leader_election.cc:302] T 00000000000000000000000000000000 P 1d95bd60277849b18746b22a96850237 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d95bd60277849b18746b22a96850237; no voters:
I20260810 18:52:38.584779 10782 leader_election.cc:288] T 00000000000000000000000000000000 P 1d95bd60277849b18746b22a96850237 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:52:38.584839 10792 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d95bd60277849b18746b22a96850237 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:52:38.585079 10792 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d95bd60277849b18746b22a96850237 [term 1 LEADER]: Becoming Leader. State: Replica: 1d95bd60277849b18746b22a96850237, State: Running, Role: LEADER
I20260810 18:52:38.585362 10792 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d95bd60277849b18746b22a96850237 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d95bd60277849b18746b22a96850237" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46693 } }
I20260810 18:52:38.585949 10782 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d95bd60277849b18746b22a96850237 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:52:38.586469 10792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d95bd60277849b18746b22a96850237 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d95bd60277849b18746b22a96850237" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d95bd60277849b18746b22a96850237" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46693 } } }
I20260810 18:52:38.586578 10792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d95bd60277849b18746b22a96850237 [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:38.586781 10792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d95bd60277849b18746b22a96850237 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d95bd60277849b18746b22a96850237. Latest consensus state: current_term: 1 leader_uuid: "1d95bd60277849b18746b22a96850237" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d95bd60277849b18746b22a96850237" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46693 } } }
I20260810 18:52:38.586933 10792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d95bd60277849b18746b22a96850237 [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:38.587280 10797 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:52:38.590323 10797 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:52:38.593438 10785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:38.584353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.129:58689 (local address 127.0.0.1:46693)
0810 18:52:38.584462 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:38.584466 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:38.584758 (+ 292us) server_negotiation.cc:407] Connection header received
0810 18:52:38.584907 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:38.584910 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:38.584957 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:38.585040 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:38.585759 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:38.586390 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:38.587536 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:38.587866 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:38.587955 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:38.589654 (+ 1699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:38.589657 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:38.589659 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:38.589689 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:38.591231 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:38.591694 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:38.591698 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:38.591700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:38.591778 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:38.592134 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:38.592137 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:38.592139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:38.592364 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:38.592499 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:38.592972 (+ 473us) server_negotiation.cc:299] Negotiation successful
0810 18:52:38.593271 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 18:52:38.597860 10797 catalog_manager.cc:1347] Generated new cluster ID: 564b697e91cc440aa40a32fdbc628fa5
I20260810 18:52:38.597921 10797 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:52:38.619073 10797 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:52:38.619946 10797 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:52:38.628738 10797 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d95bd60277849b18746b22a96850237: Generated new TSK 0
I20260810 18:52:38.629432 10797 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:52:38.633297 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 10707
W20260810 18:52:38.643442 10690 connection.cc:535] client connection to 127.0.0.1:46693 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46693): Connection reset by peer (error 104)
2026-08-10T18:52:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1482 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:52:38 dist-test-slave-cbsf krb5kdc[10816](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:52:38 dist-test-slave-cbsf krb5kdc[10816](info): set up 2 sockets
Aug 10 18:52:38 dist-test-slave-cbsf krb5kdc[10816](info): commencing operation
krb5kdc: starting...
W20260810 18:52:39.743053 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:52:39 dist-test-slave-cbsf krb5kdc[10816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387959, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:52:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:52:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:52:39.872100 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40803
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40803
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:52:39.982764 10833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:39.983012 10833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:39.983052 10833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:39.987124 10833 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:52:39.987179 10833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:39.987198 10833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:39.987214 10833 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:52:39.987229 10833 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:52:39.993461 10833 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40803
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:39.994410 10833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:39.995373 10833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:40.003902 10839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:40.003963 10840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:40.003949 10842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:40.004268 10833 server_base.cc:1034] running on GCE node
I20260810 18:52:40.004902 10833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:40.006000 10833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:40.007164 10833 hybrid_clock.cc:648] HybridClock initialized: now 1786387960007146 us; error 29 us; skew 500 ppm
Aug 10 18:52:40 dist-test-slave-cbsf krb5kdc[10816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387960, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:40.009873 10833 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:52:40.010692 10833 webserver.cc:460] Webserver started at http://127.0.0.1:41791/ using document root <none> and password file <none>
I20260810 18:52:40.011186 10833 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:40.011241 10833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:40.011397 10833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:40.013406 10833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "ae2ae7fdb1d5423e85edc1407c87713e"
format_stamp: "Formatted at 2026-08-10 18:52:40 on dist-test-slave-cbsf"
server_key: "f4333912c49592e88bd0d6473e719100"
server_key_iv: "da6e6b6d98a7f0768e415fc8fe3930ba"
server_key_version: "encryptionkey@0"
I20260810 18:52:40.013880 10833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "ae2ae7fdb1d5423e85edc1407c87713e"
format_stamp: "Formatted at 2026-08-10 18:52:40 on dist-test-slave-cbsf"
server_key: "f4333912c49592e88bd0d6473e719100"
server_key_iv: "da6e6b6d98a7f0768e415fc8fe3930ba"
server_key_version: "encryptionkey@0"
I20260810 18:52:40.017444 10833 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:52:40.019649 10850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:40.020876 10833 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:52:40.020982 10833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "ae2ae7fdb1d5423e85edc1407c87713e"
format_stamp: "Formatted at 2026-08-10 18:52:40 on dist-test-slave-cbsf"
server_key: "f4333912c49592e88bd0d6473e719100"
server_key_iv: "da6e6b6d98a7f0768e415fc8fe3930ba"
server_key_version: "encryptionkey@0"
I20260810 18:52:40.021060 10833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:40.043526 10833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:40.046317 10833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:40.046466 10833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:40.054154 10833 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40803
I20260810 18:52:40.054164 10906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40803 every 8 connection(s)
I20260810 18:52:40.055135 10833 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:52:40.058077 10907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:40.059145 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 10833
I20260810 18:52:40.059289 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:52:40.059643 19106 external_mini_cluster.cc:1428] Setting key de191338eebfb8c2a1fafc6d145bbb2a
I20260810 18:52:40.064456 10907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae2ae7fdb1d5423e85edc1407c87713e: Bootstrap starting.
I20260810 18:52:40.067158 10907 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae2ae7fdb1d5423e85edc1407c87713e: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:40.067958 10907 log.cc:824] T 00000000000000000000000000000000 P ae2ae7fdb1d5423e85edc1407c87713e: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:40.068516 10910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:40.061059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.129:42683 (local address 127.0.0.1:40803)
0810 18:52:40.061487 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:40.061502 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:40.061538 (+ 36us) server_negotiation.cc:407] Connection header received
0810 18:52:40.062151 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:40.062167 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:40.062496 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:40.062757 (+ 261us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:40.063103 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:40.064048 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:40.065274 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:40.065746 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:40.065879 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:40.066766 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:40.066770 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:40.066783 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:52:40.067144 (+ 361us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:40.067196 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:52:40.067254 (+ 58us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:40.067412 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:40.067608 (+ 196us) server_negotiation.cc:299] Negotiation successful
0810 18:52:40.067734 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":137,"threads_started":1}
I20260810 18:52:40.070166 10907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae2ae7fdb1d5423e85edc1407c87713e: No bootstrap required, opened a new log
I20260810 18:52:40.071587 19106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:52:40.074137 10907 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae2ae7fdb1d5423e85edc1407c87713e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae2ae7fdb1d5423e85edc1407c87713e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40803 } }
I20260810 18:52:40.074430 10907 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae2ae7fdb1d5423e85edc1407c87713e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:40.074504 10907 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae2ae7fdb1d5423e85edc1407c87713e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae2ae7fdb1d5423e85edc1407c87713e, State: Initialized, Role: FOLLOWER
I20260810 18:52:40.074945 10907 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae2ae7fdb1d5423e85edc1407c87713e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae2ae7fdb1d5423e85edc1407c87713e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40803 } }
I20260810 18:52:40.075109 10907 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae2ae7fdb1d5423e85edc1407c87713e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:52:40.075176 10907 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae2ae7fdb1d5423e85edc1407c87713e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:52:40.075294 10907 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae2ae7fdb1d5423e85edc1407c87713e [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:40.076463 10907 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae2ae7fdb1d5423e85edc1407c87713e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae2ae7fdb1d5423e85edc1407c87713e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40803 } }
I20260810 18:52:40.076870 10907 leader_election.cc:302] T 00000000000000000000000000000000 P ae2ae7fdb1d5423e85edc1407c87713e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae2ae7fdb1d5423e85edc1407c87713e; no voters:
I20260810 18:52:40.077247 10907 leader_election.cc:288] T 00000000000000000000000000000000 P ae2ae7fdb1d5423e85edc1407c87713e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:52:40.077386 10917 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae2ae7fdb1d5423e85edc1407c87713e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:52:40.077706 10917 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae2ae7fdb1d5423e85edc1407c87713e [term 1 LEADER]: Becoming Leader. State: Replica: ae2ae7fdb1d5423e85edc1407c87713e, State: Running, Role: LEADER
I20260810 18:52:40.078092 10917 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae2ae7fdb1d5423e85edc1407c87713e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae2ae7fdb1d5423e85edc1407c87713e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40803 } }
I20260810 18:52:40.078392 10910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:40.073676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.129:52585 (local address 127.0.0.1:40803)
0810 18:52:40.073804 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:40.073808 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:40.073950 (+ 142us) server_negotiation.cc:407] Connection header received
0810 18:52:40.074041 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:40.074044 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:40.074088 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:40.074168 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:40.074402 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:40.075148 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:40.076362 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:40.076669 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:40.076810 (+ 141us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:40.077730 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:40.077735 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:40.077738 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:52:40.077751 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:40.077782 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:52:40.077797 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:40.077886 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:40.078238 (+ 352us) server_negotiation.cc:299] Negotiation successful
0810 18:52:40.078272 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 18:52:40.078557 10907 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae2ae7fdb1d5423e85edc1407c87713e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:52:40.079945 10919 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae2ae7fdb1d5423e85edc1407c87713e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae2ae7fdb1d5423e85edc1407c87713e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae2ae7fdb1d5423e85edc1407c87713e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40803 } } }
I20260810 18:52:40.080102 10919 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae2ae7fdb1d5423e85edc1407c87713e [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:40.080328 10919 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae2ae7fdb1d5423e85edc1407c87713e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae2ae7fdb1d5423e85edc1407c87713e. Latest consensus state: current_term: 1 leader_uuid: "ae2ae7fdb1d5423e85edc1407c87713e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae2ae7fdb1d5423e85edc1407c87713e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40803 } } }
I20260810 18:52:40.080420 10919 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae2ae7fdb1d5423e85edc1407c87713e [sys.catalog]: This master's current role is: LEADER
W20260810 18:52:40.083197 10935 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ae2ae7fdb1d5423e85edc1407c87713e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:52:40.083277 10935 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:52:40.083340 10934 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:52:40.084052 10934 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:52:40.089443 10934 catalog_manager.cc:1347] Generated new cluster ID: 1490572af6b3474a89e82b97475e154b
I20260810 18:52:40.089531 10934 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:52:40.102501 10934 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:52:40.103436 10934 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:52:40.113207 10934 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae2ae7fdb1d5423e85edc1407c87713e: Generated new TSK 0
I20260810 18:52:40.114013 10934 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:52:40.119606 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 10833
W20260810 18:52:40.129791 10814 connection.cc:535] client connection to 127.0.0.1:40803 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40803): Connection reset by peer (error 104)
2026-08-10T18:52:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1486 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:52:40 dist-test-slave-cbsf krb5kdc[10942](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:52:40 dist-test-slave-cbsf krb5kdc[10942](info): set up 2 sockets
Aug 10 18:52:40 dist-test-slave-cbsf krb5kdc[10942](info): commencing operation
krb5kdc: starting...
W20260810 18:52:41.264482 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:52:41 dist-test-slave-cbsf krb5kdc[10942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387961, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:52:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:52:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:52:41.392006 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40751
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40751
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:52:41.501425 10957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:41.501665 10957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:41.501704 10957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:41.505687 10957 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:52:41.505739 10957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:41.505759 10957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:41.505774 10957 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:52:41.505825 10957 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:52:41.510900 10957 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40751
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:41.511883 10957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:41.512913 10957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:41.521122 10965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:41.521191 10966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:41.521365 10957 server_base.cc:1034] running on GCE node
W20260810 18:52:41.521343 10968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:41.521966 10957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:41.523000 10957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:41.524181 10957 hybrid_clock.cc:648] HybridClock initialized: now 1786387961524169 us; error 32 us; skew 500 ppm
Aug 10 18:52:41 dist-test-slave-cbsf krb5kdc[10942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387961, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:41.526825 10957 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:52:41.527709 10957 webserver.cc:460] Webserver started at http://127.0.0.1:45547/ using document root <none> and password file <none>
I20260810 18:52:41.528249 10957 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:41.528337 10957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:41.528576 10957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:41.530512 10957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "fcfdfbeebc9f435291319480fc3b5fa1"
format_stamp: "Formatted at 2026-08-10 18:52:41 on dist-test-slave-cbsf"
server_key: "e807df61cce49a79df71697acc60b634"
server_key_iv: "b76ab73d231ed16bc2d4cd511663f06c"
server_key_version: "encryptionkey@0"
I20260810 18:52:41.531064 10957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "fcfdfbeebc9f435291319480fc3b5fa1"
format_stamp: "Formatted at 2026-08-10 18:52:41 on dist-test-slave-cbsf"
server_key: "e807df61cce49a79df71697acc60b634"
server_key_iv: "b76ab73d231ed16bc2d4cd511663f06c"
server_key_version: "encryptionkey@0"
I20260810 18:52:41.534633 10957 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 18:52:41.536911 10976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:41.538130 10957 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 18:52:41.538229 10957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "fcfdfbeebc9f435291319480fc3b5fa1"
format_stamp: "Formatted at 2026-08-10 18:52:41 on dist-test-slave-cbsf"
server_key: "e807df61cce49a79df71697acc60b634"
server_key_iv: "b76ab73d231ed16bc2d4cd511663f06c"
server_key_version: "encryptionkey@0"
I20260810 18:52:41.538307 10957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:41.542569 10957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:41.545339 10957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:41.545488 10957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:41.552011 10957 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40751
I20260810 18:52:41.552090 11031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40751 every 8 connection(s)
I20260810 18:52:41.552980 10957 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:52:41.555882 11032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:41.557827 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 10957
I20260810 18:52:41.557937 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:52:41.558245 19106 external_mini_cluster.cc:1428] Setting key c22df54be6ceb053f55b4350e64a9c1e
I20260810 18:52:41.562350 11032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcfdfbeebc9f435291319480fc3b5fa1: Bootstrap starting.
I20260810 18:52:41.563215 11035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:41.559547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.129:35469 (local address 127.0.0.1:40751)
0810 18:52:41.559991 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:41.560005 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:41.560048 (+ 43us) server_negotiation.cc:407] Connection header received
0810 18:52:41.560648 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:41.560663 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:41.560986 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:41.561297 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:41.562092 (+ 795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:41.562097 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:41.562108 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:52:41.562118 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:41.562172 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:52:41.562191 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:41.562357 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:41.562428 (+ 71us) server_negotiation.cc:299] Negotiation successful
0810 18:52:41.562583 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":144,"threads_started":1}
I20260810 18:52:41.564838 11032 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fcfdfbeebc9f435291319480fc3b5fa1: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:41.565721 11032 log.cc:824] T 00000000000000000000000000000000 P fcfdfbeebc9f435291319480fc3b5fa1: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:41.566013 19106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:52:41.567783 11032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcfdfbeebc9f435291319480fc3b5fa1: No bootstrap required, opened a new log
I20260810 18:52:41.569851 11035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:41.568189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.129:42825 (local address 127.0.0.1:40751)
0810 18:52:41.568297 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:41.568303 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:41.568488 (+ 185us) server_negotiation.cc:407] Connection header received
0810 18:52:41.568624 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:41.568626 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:41.568651 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:41.568781 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:41.569480 (+ 699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:41.569483 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:41.569485 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:52:41.569494 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:41.569516 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:52:41.569526 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:41.569587 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:41.569700 (+ 113us) server_negotiation.cc:299] Negotiation successful
0810 18:52:41.569729 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 18:52:41.572315 11032 raft_consensus.cc:348] T 00000000000000000000000000000000 P fcfdfbeebc9f435291319480fc3b5fa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcfdfbeebc9f435291319480fc3b5fa1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40751 } }
I20260810 18:52:41.572578 11032 raft_consensus.cc:374] T 00000000000000000000000000000000 P fcfdfbeebc9f435291319480fc3b5fa1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:41.572644 11032 raft_consensus.cc:729] T 00000000000000000000000000000000 P fcfdfbeebc9f435291319480fc3b5fa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcfdfbeebc9f435291319480fc3b5fa1, State: Initialized, Role: FOLLOWER
I20260810 18:52:41.573112 11032 consensus_queue.cc:260] T 00000000000000000000000000000000 P fcfdfbeebc9f435291319480fc3b5fa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcfdfbeebc9f435291319480fc3b5fa1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40751 } }
I20260810 18:52:41.573284 11032 raft_consensus.cc:388] T 00000000000000000000000000000000 P fcfdfbeebc9f435291319480fc3b5fa1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:52:41.573359 11032 raft_consensus.cc:482] T 00000000000000000000000000000000 P fcfdfbeebc9f435291319480fc3b5fa1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:52:41.573477 11032 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcfdfbeebc9f435291319480fc3b5fa1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:41.574425 11032 raft_consensus.cc:504] T 00000000000000000000000000000000 P fcfdfbeebc9f435291319480fc3b5fa1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcfdfbeebc9f435291319480fc3b5fa1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40751 } }
I20260810 18:52:41.574770 11032 leader_election.cc:302] T 00000000000000000000000000000000 P fcfdfbeebc9f435291319480fc3b5fa1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fcfdfbeebc9f435291319480fc3b5fa1; no voters:
I20260810 18:52:41.575116 11032 leader_election.cc:288] T 00000000000000000000000000000000 P fcfdfbeebc9f435291319480fc3b5fa1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:52:41.575213 11044 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fcfdfbeebc9f435291319480fc3b5fa1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:52:41.575457 11044 raft_consensus.cc:686] T 00000000000000000000000000000000 P fcfdfbeebc9f435291319480fc3b5fa1 [term 1 LEADER]: Becoming Leader. State: Replica: fcfdfbeebc9f435291319480fc3b5fa1, State: Running, Role: LEADER
I20260810 18:52:41.575870 11044 consensus_queue.cc:237] T 00000000000000000000000000000000 P fcfdfbeebc9f435291319480fc3b5fa1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcfdfbeebc9f435291319480fc3b5fa1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40751 } }
I20260810 18:52:41.576380 11032 sys_catalog.cc:564] T 00000000000000000000000000000000 P fcfdfbeebc9f435291319480fc3b5fa1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:52:41.577638 11046 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcfdfbeebc9f435291319480fc3b5fa1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fcfdfbeebc9f435291319480fc3b5fa1. Latest consensus state: current_term: 1 leader_uuid: "fcfdfbeebc9f435291319480fc3b5fa1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcfdfbeebc9f435291319480fc3b5fa1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40751 } } }
I20260810 18:52:41.577764 11046 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcfdfbeebc9f435291319480fc3b5fa1 [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:41.578178 11045 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcfdfbeebc9f435291319480fc3b5fa1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fcfdfbeebc9f435291319480fc3b5fa1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcfdfbeebc9f435291319480fc3b5fa1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40751 } } }
I20260810 18:52:41.578822 11054 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:52:41.578961 11045 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcfdfbeebc9f435291319480fc3b5fa1 [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:41.581576 11054 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:52:41.586397 11054 catalog_manager.cc:1347] Generated new cluster ID: 3daee8e3299d492ab2fcd7240c33d669
I20260810 18:52:41.586464 11054 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:52:41.594069 11054 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:52:41.594339 11054 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:52:41.602067 11054 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fcfdfbeebc9f435291319480fc3b5fa1: Generated new TSK 0
I20260810 18:52:41.602656 11054 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:52:41.612880 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 10957
2026-08-10T18:52:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1491 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:52:41 dist-test-slave-cbsf krb5kdc[11069](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:52:41 dist-test-slave-cbsf krb5kdc[11069](info): set up 2 sockets
Aug 10 18:52:41 dist-test-slave-cbsf krb5kdc[11069](info): commencing operation
krb5kdc: starting...
W20260810 18:52:42.725921 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:52:42 dist-test-slave-cbsf krb5kdc[11069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:52:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:52:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:52:42.851220 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40437
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40437
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:52:42.960021 11084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:42.960246 11084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:42.960284 11084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:42.964186 11084 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:52:42.964239 11084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:42.964258 11084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:42.964274 11084 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:52:42.964288 11084 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:52:42.969218 11084 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40437
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:42.970075 11084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:42.970955 11084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:42.979264 11091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:42.979259 11089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:42.979549 11084 server_base.cc:1034] running on GCE node
W20260810 18:52:42.979259 11093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:42.980195 11084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:42.981278 11084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:42.982438 11084 hybrid_clock.cc:648] HybridClock initialized: now 1786387962982420 us; error 28 us; skew 500 ppm
Aug 10 18:52:42 dist-test-slave-cbsf krb5kdc[11069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387962, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:42.985306 11084 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:52:42.986227 11084 webserver.cc:460] Webserver started at http://127.0.0.1:37821/ using document root <none> and password file <none>
I20260810 18:52:42.986920 11084 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:42.986989 11084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:42.987226 11084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:42.989470 11084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "de0940742cc64abca7af58e8466bbbd3"
format_stamp: "Formatted at 2026-08-10 18:52:42 on dist-test-slave-cbsf"
server_key: "15767de50e18c9f4084ed2008f0c5060"
server_key_iv: "368622b2f52b6bda25d422509afcf130"
server_key_version: "encryptionkey@0"
I20260810 18:52:42.990094 11084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "de0940742cc64abca7af58e8466bbbd3"
format_stamp: "Formatted at 2026-08-10 18:52:42 on dist-test-slave-cbsf"
server_key: "15767de50e18c9f4084ed2008f0c5060"
server_key_iv: "368622b2f52b6bda25d422509afcf130"
server_key_version: "encryptionkey@0"
I20260810 18:52:42.995211 11084 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 18:52:42.998723 11101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:43.000483 11084 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:52:43.000615 11084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "de0940742cc64abca7af58e8466bbbd3"
format_stamp: "Formatted at 2026-08-10 18:52:42 on dist-test-slave-cbsf"
server_key: "15767de50e18c9f4084ed2008f0c5060"
server_key_iv: "368622b2f52b6bda25d422509afcf130"
server_key_version: "encryptionkey@0"
I20260810 18:52:43.000764 11084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:43.032047 11084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:43.035019 11084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:43.035182 11084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:43.043223 11084 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40437
I20260810 18:52:43.043254 11159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40437 every 8 connection(s)
I20260810 18:52:43.044214 11084 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:52:43.046989 11160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:43.047672 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 11084
I20260810 18:52:43.047830 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:52:43.048161 19106 external_mini_cluster.cc:1428] Setting key 3f5c57cf2432e3de2264f82aa5267a4a
I20260810 18:52:43.053565 11160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de0940742cc64abca7af58e8466bbbd3: Bootstrap starting.
I20260810 18:52:43.056372 11160 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de0940742cc64abca7af58e8466bbbd3: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:43.057313 11160 log.cc:824] T 00000000000000000000000000000000 P de0940742cc64abca7af58e8466bbbd3: Log is configured to *not* fsync() on all Append() calls
Aug 10 18:52:43 dist-test-slave-cbsf krb5kdc[11069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 18:52:43.059278 11160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de0940742cc64abca7af58e8466bbbd3: No bootstrap required, opened a new log
I20260810 18:52:43.063174 11163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:43.049667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.129:42649 (local address 127.0.0.1:40437)
0810 18:52:43.050118 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:43.050134 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:43.050169 (+ 35us) server_negotiation.cc:407] Connection header received
0810 18:52:43.050792 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:43.050807 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:43.051138 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:43.051411 (+ 273us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:43.052393 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:43.053423 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:43.054599 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:43.055041 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:43.055160 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:43.058167 (+ 3007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:43.058172 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:43.058186 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:43.058212 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:43.059980 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:43.060559 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:43.060564 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:43.060568 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:43.060676 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:43.061082 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:43.061084 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:43.061086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:43.061460 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:43.061661 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:43.062180 (+ 519us) server_negotiation.cc:299] Negotiation successful
0810 18:52:43.062438 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":130,"threads_started":1}
I20260810 18:52:43.063587 11160 raft_consensus.cc:348] T 00000000000000000000000000000000 P de0940742cc64abca7af58e8466bbbd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de0940742cc64abca7af58e8466bbbd3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40437 } }
I20260810 18:52:43.063832 11160 raft_consensus.cc:374] T 00000000000000000000000000000000 P de0940742cc64abca7af58e8466bbbd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:43.063895 11160 raft_consensus.cc:729] T 00000000000000000000000000000000 P de0940742cc64abca7af58e8466bbbd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de0940742cc64abca7af58e8466bbbd3, State: Initialized, Role: FOLLOWER
I20260810 18:52:43.064378 11160 consensus_queue.cc:260] T 00000000000000000000000000000000 P de0940742cc64abca7af58e8466bbbd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de0940742cc64abca7af58e8466bbbd3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40437 } }
I20260810 18:52:43.064539 11160 raft_consensus.cc:388] T 00000000000000000000000000000000 P de0940742cc64abca7af58e8466bbbd3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:52:43.064603 11160 raft_consensus.cc:482] T 00000000000000000000000000000000 P de0940742cc64abca7af58e8466bbbd3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:52:43.064743 11160 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de0940742cc64abca7af58e8466bbbd3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:43.065816 11160 raft_consensus.cc:504] T 00000000000000000000000000000000 P de0940742cc64abca7af58e8466bbbd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de0940742cc64abca7af58e8466bbbd3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40437 } }
I20260810 18:52:43.066175 11160 leader_election.cc:302] T 00000000000000000000000000000000 P de0940742cc64abca7af58e8466bbbd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de0940742cc64abca7af58e8466bbbd3; no voters:
I20260810 18:52:43.066340 19106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:52:43.066516 11160 leader_election.cc:288] T 00000000000000000000000000000000 P de0940742cc64abca7af58e8466bbbd3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:52:43.066644 11166 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de0940742cc64abca7af58e8466bbbd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:52:43.066872 11166 raft_consensus.cc:686] T 00000000000000000000000000000000 P de0940742cc64abca7af58e8466bbbd3 [term 1 LEADER]: Becoming Leader. State: Replica: de0940742cc64abca7af58e8466bbbd3, State: Running, Role: LEADER
I20260810 18:52:43.067189 11166 consensus_queue.cc:237] T 00000000000000000000000000000000 P de0940742cc64abca7af58e8466bbbd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de0940742cc64abca7af58e8466bbbd3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40437 } }
I20260810 18:52:43.067852 11160 sys_catalog.cc:564] T 00000000000000000000000000000000 P de0940742cc64abca7af58e8466bbbd3 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:52:43.068964 11168 sys_catalog.cc:455] T 00000000000000000000000000000000 P de0940742cc64abca7af58e8466bbbd3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de0940742cc64abca7af58e8466bbbd3. Latest consensus state: current_term: 1 leader_uuid: "de0940742cc64abca7af58e8466bbbd3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de0940742cc64abca7af58e8466bbbd3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40437 } } }
I20260810 18:52:43.069115 11168 sys_catalog.cc:458] T 00000000000000000000000000000000 P de0940742cc64abca7af58e8466bbbd3 [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:43.069413 11167 sys_catalog.cc:455] T 00000000000000000000000000000000 P de0940742cc64abca7af58e8466bbbd3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de0940742cc64abca7af58e8466bbbd3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de0940742cc64abca7af58e8466bbbd3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40437 } } }
I20260810 18:52:43.069502 11167 sys_catalog.cc:458] T 00000000000000000000000000000000 P de0940742cc64abca7af58e8466bbbd3 [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:43.069842 11180 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:52:43.073272 11180 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:52:43.078431 11163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:43.068680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.129:32897 (local address 127.0.0.1:40437)
0810 18:52:43.068819 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:43.068824 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:43.069096 (+ 272us) server_negotiation.cc:407] Connection header received
0810 18:52:43.069306 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:43.069312 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:43.069368 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:43.069438 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:43.070553 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:43.071268 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:43.072361 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:43.072684 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:43.072809 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:43.074415 (+ 1606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:43.074420 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:43.074423 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:43.074449 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:43.076151 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:43.076883 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:43.076889 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:43.076893 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:43.076998 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:43.077479 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:43.077482 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:43.077484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:43.077735 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:43.077898 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:43.078182 (+ 284us) server_negotiation.cc:299] Negotiation successful
0810 18:52:43.078287 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 18:52:43.079361 11180 catalog_manager.cc:1347] Generated new cluster ID: 4279ae4ad4154cc5bdc7473322aabb75
I20260810 18:52:43.079470 11180 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:52:43.103914 11180 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:52:43.105041 11180 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:52:43.116032 11180 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de0940742cc64abca7af58e8466bbbd3: Generated new TSK 0
I20260810 18:52:43.116870 11180 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:52:43.125765 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 11084
W20260810 18:52:43.134917 11067 connection.cc:535] client connection to 127.0.0.1:40437 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40437): Connection reset by peer (error 104)
2026-08-10T18:52:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1512 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:52:43 dist-test-slave-cbsf krb5kdc[11194](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:52:43 dist-test-slave-cbsf krb5kdc[11194](info): set up 2 sockets
Aug 10 18:52:43 dist-test-slave-cbsf krb5kdc[11194](info): commencing operation
krb5kdc: starting...
W20260810 18:52:44.215076 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:52:44 dist-test-slave-cbsf krb5kdc[11194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387964, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:52:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:52:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:52:44.336030 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38691
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38691
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:52:44.444617 11211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:44.444849 11211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:44.444885 11211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:44.448544 11211 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:52:44.448586 11211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:44.448608 11211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:44.448626 11211 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:52:44.448659 11211 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:52:44.454342 11211 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38691
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:44.455291 11211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:44.456250 11211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:44.464612 11220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:44.464646 11218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:44.464612 11222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:44.465031 11211 server_base.cc:1034] running on GCE node
I20260810 18:52:44.465523 11211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:44.466518 11211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:44.467701 11211 hybrid_clock.cc:648] HybridClock initialized: now 1786387964467685 us; error 30 us; skew 500 ppm
Aug 10 18:52:44 dist-test-slave-cbsf krb5kdc[11194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387964, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:44.474578 11211 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:52:44.475399 11211 webserver.cc:460] Webserver started at http://127.0.0.1:46423/ using document root <none> and password file <none>
I20260810 18:52:44.475924 11211 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:44.476012 11211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:44.476249 11211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:44.478329 11211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "eef2e1617a35478db0c409d50c4a525f"
format_stamp: "Formatted at 2026-08-10 18:52:44 on dist-test-slave-cbsf"
server_key: "73be649b40578c82063ef4de1e8ea7b0"
server_key_iv: "ea5211bbe019bee66bcab16ca5291ee8"
server_key_version: "encryptionkey@0"
I20260810 18:52:44.478884 11211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "eef2e1617a35478db0c409d50c4a525f"
format_stamp: "Formatted at 2026-08-10 18:52:44 on dist-test-slave-cbsf"
server_key: "73be649b40578c82063ef4de1e8ea7b0"
server_key_iv: "ea5211bbe019bee66bcab16ca5291ee8"
server_key_version: "encryptionkey@0"
I20260810 18:52:44.482524 11211 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 18:52:44.484923 11229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:44.486063 11211 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:52:44.486203 11211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "eef2e1617a35478db0c409d50c4a525f"
format_stamp: "Formatted at 2026-08-10 18:52:44 on dist-test-slave-cbsf"
server_key: "73be649b40578c82063ef4de1e8ea7b0"
server_key_iv: "ea5211bbe019bee66bcab16ca5291ee8"
server_key_version: "encryptionkey@0"
I20260810 18:52:44.486312 11211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:44.501883 11211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:44.504666 11211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:44.504900 11211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:44.512516 11211 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38691
I20260810 18:52:44.512552 11287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38691 every 8 connection(s)
I20260810 18:52:44.513473 11211 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:52:44.516134 11288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:44.521613 11288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eef2e1617a35478db0c409d50c4a525f: Bootstrap starting.
I20260810 18:52:44.521927 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 11211
I20260810 18:52:44.522070 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:52:44.522343 19106 external_mini_cluster.cc:1428] Setting key 59944eb16a7da6a82c14def434a48d9a
I20260810 18:52:44.524219 11288 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eef2e1617a35478db0c409d50c4a525f: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:44.525017 11288 log.cc:824] T 00000000000000000000000000000000 P eef2e1617a35478db0c409d50c4a525f: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:44.526886 11288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eef2e1617a35478db0c409d50c4a525f: No bootstrap required, opened a new log
I20260810 18:52:44.530339 11288 raft_consensus.cc:348] T 00000000000000000000000000000000 P eef2e1617a35478db0c409d50c4a525f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef2e1617a35478db0c409d50c4a525f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38691 } }
I20260810 18:52:44.530629 11288 raft_consensus.cc:374] T 00000000000000000000000000000000 P eef2e1617a35478db0c409d50c4a525f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:44.530717 11288 raft_consensus.cc:729] T 00000000000000000000000000000000 P eef2e1617a35478db0c409d50c4a525f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eef2e1617a35478db0c409d50c4a525f, State: Initialized, Role: FOLLOWER
I20260810 18:52:44.531188 11288 consensus_queue.cc:260] T 00000000000000000000000000000000 P eef2e1617a35478db0c409d50c4a525f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef2e1617a35478db0c409d50c4a525f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38691 } }
I20260810 18:52:44.531306 11288 raft_consensus.cc:388] T 00000000000000000000000000000000 P eef2e1617a35478db0c409d50c4a525f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:52:44.531342 11291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:44.523670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.129:33975 (local address 127.0.0.1:38691)
0810 18:52:44.524095 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:44.524110 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:44.524144 (+ 34us) server_negotiation.cc:407] Connection header received
0810 18:52:44.524926 (+ 782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:44.524945 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:44.525316 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:44.525630 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:44.526055 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:44.527095 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:44.528008 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:44.528469 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:44.528643 (+ 174us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:44.529600 (+ 957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:44.529606 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:44.529618 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:52:44.529969 (+ 351us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:44.530030 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:52:44.530054 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:44.530195 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:44.530379 (+ 184us) server_negotiation.cc:299] Negotiation successful
0810 18:52:44.530508 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":124,"threads_started":1}
I20260810 18:52:44.531384 11288 raft_consensus.cc:482] T 00000000000000000000000000000000 P eef2e1617a35478db0c409d50c4a525f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:52:44.531476 11288 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eef2e1617a35478db0c409d50c4a525f [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:44.532269 11288 raft_consensus.cc:504] T 00000000000000000000000000000000 P eef2e1617a35478db0c409d50c4a525f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef2e1617a35478db0c409d50c4a525f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38691 } }
I20260810 18:52:44.532641 11288 leader_election.cc:302] T 00000000000000000000000000000000 P eef2e1617a35478db0c409d50c4a525f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eef2e1617a35478db0c409d50c4a525f; no voters:
I20260810 18:52:44.533004 11288 leader_election.cc:288] T 00000000000000000000000000000000 P eef2e1617a35478db0c409d50c4a525f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:52:44.533062 11293 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eef2e1617a35478db0c409d50c4a525f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:52:44.533244 11293 raft_consensus.cc:686] T 00000000000000000000000000000000 P eef2e1617a35478db0c409d50c4a525f [term 1 LEADER]: Becoming Leader. State: Replica: eef2e1617a35478db0c409d50c4a525f, State: Running, Role: LEADER
I20260810 18:52:44.533505 11293 consensus_queue.cc:237] T 00000000000000000000000000000000 P eef2e1617a35478db0c409d50c4a525f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef2e1617a35478db0c409d50c4a525f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38691 } }
I20260810 18:52:44.534202 11288 sys_catalog.cc:564] T 00000000000000000000000000000000 P eef2e1617a35478db0c409d50c4a525f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:52:44.534293 19106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:52:44.535030 11294 sys_catalog.cc:455] T 00000000000000000000000000000000 P eef2e1617a35478db0c409d50c4a525f [sys.catalog]: SysCatalogTable state changed. Reason: New leader eef2e1617a35478db0c409d50c4a525f. Latest consensus state: current_term: 1 leader_uuid: "eef2e1617a35478db0c409d50c4a525f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef2e1617a35478db0c409d50c4a525f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38691 } } }
I20260810 18:52:44.535168 11294 sys_catalog.cc:458] T 00000000000000000000000000000000 P eef2e1617a35478db0c409d50c4a525f [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:44.535490 11295 sys_catalog.cc:455] T 00000000000000000000000000000000 P eef2e1617a35478db0c409d50c4a525f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eef2e1617a35478db0c409d50c4a525f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef2e1617a35478db0c409d50c4a525f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38691 } } }
I20260810 18:52:44.535576 11295 sys_catalog.cc:458] T 00000000000000000000000000000000 P eef2e1617a35478db0c409d50c4a525f [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:44.535849 11302 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:52:44.538923 11302 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:52:44.541720 11291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:44.536918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.129:36125 (local address 127.0.0.1:38691)
0810 18:52:44.537037 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:44.537043 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:44.537385 (+ 342us) server_negotiation.cc:407] Connection header received
0810 18:52:44.537529 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:44.537532 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:44.537585 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:44.537665 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:44.537942 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:44.538719 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:44.539805 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:44.540142 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:44.540255 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:44.541195 (+ 940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:44.541199 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:44.541202 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:52:44.541215 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:44.541261 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:52:44.541274 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:44.541373 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:44.541553 (+ 180us) server_negotiation.cc:299] Negotiation successful
0810 18:52:44.541588 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 18:52:44.544322 11302 catalog_manager.cc:1347] Generated new cluster ID: d958bed122014d23aab70deaa661238a
I20260810 18:52:44.544381 11302 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:52:44.550828 11302 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:52:44.551609 11302 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:52:44.555250 11302 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eef2e1617a35478db0c409d50c4a525f: Generated new TSK 0
I20260810 18:52:44.555902 11302 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:52:44.560815 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 11211
W20260810 18:52:44.570216 11192 connection.cc:535] client connection to 127.0.0.1:38691 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38691): Connection reset by peer (error 104)
2026-08-10T18:52:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1435 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:52:44 dist-test-slave-cbsf krb5kdc[11321](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:52:44 dist-test-slave-cbsf krb5kdc[11321](info): set up 2 sockets
Aug 10 18:52:44 dist-test-slave-cbsf krb5kdc[11321](info): commencing operation
krb5kdc: starting...
W20260810 18:52:45.673893 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:52:45 dist-test-slave-cbsf krb5kdc[11321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:52:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:52:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:52:45.797905 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42293
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42293
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:52:45.907228 11335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:45.907456 11335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:45.907507 11335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:45.911317 11335 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:52:45.911370 11335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:45.911389 11335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:45.911405 11335 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:52:45.911419 11335 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:52:45.916441 11335 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42293
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387687857264-19106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:45.917415 11335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:45.918313 11335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:45.926671 11344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:45.926765 11335 server_base.cc:1034] running on GCE node
W20260810 18:52:45.926681 11340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:45.926677 11341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:45.927567 11335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:45.928598 11335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:45.929778 11335 hybrid_clock.cc:648] HybridClock initialized: now 1786387965929758 us; error 44 us; skew 500 ppm
Aug 10 18:52:45 dist-test-slave-cbsf krb5kdc[11321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387965, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:45.932399 11335 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:52:45.933369 11335 webserver.cc:460] Webserver started at http://127.0.0.1:41745/ using document root <none> and password file <none>
I20260810 18:52:45.933914 11335 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:45.934002 11335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:45.934212 11335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:45.936161 11335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "88d2e05bb4af48e89d7db64018cbb3a4"
format_stamp: "Formatted at 2026-08-10 18:52:45 on dist-test-slave-cbsf"
server_key: "98ac97f66f4b6880ae6828a730e39f95"
server_key_iv: "b66d70ac7908e48e467ba1160fd1cdc3"
server_key_version: "encryptionkey@0"
I20260810 18:52:45.936791 11335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "88d2e05bb4af48e89d7db64018cbb3a4"
format_stamp: "Formatted at 2026-08-10 18:52:45 on dist-test-slave-cbsf"
server_key: "98ac97f66f4b6880ae6828a730e39f95"
server_key_iv: "b66d70ac7908e48e467ba1160fd1cdc3"
server_key_version: "encryptionkey@0"
I20260810 18:52:45.940326 11335 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:52:45.942636 11353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:45.943861 11335 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:52:45.943995 11335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "88d2e05bb4af48e89d7db64018cbb3a4"
format_stamp: "Formatted at 2026-08-10 18:52:45 on dist-test-slave-cbsf"
server_key: "98ac97f66f4b6880ae6828a730e39f95"
server_key_iv: "b66d70ac7908e48e467ba1160fd1cdc3"
server_key_version: "encryptionkey@0"
I20260810 18:52:45.944103 11335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:45.948930 11335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:45.951984 11335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:45.952207 11335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:45.960397 11335 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42293
I20260810 18:52:45.960404 11413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42293 every 8 connection(s)
I20260810 18:52:45.961431 11335 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:52:45.963968 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 11335
I20260810 18:52:45.964169 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:52:45.964366 11414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:45.964569 19106 external_mini_cluster.cc:1428] Setting key b286bddc456142aa8442028d1ac9b5bf
I20260810 18:52:45.970320 11417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:45.966008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.129:55953 (local address 127.0.0.1:42293)
0810 18:52:45.966461 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:45.966478 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:45.966518 (+ 40us) server_negotiation.cc:407] Connection header received
0810 18:52:45.967579 (+ 1061us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:45.967600 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:45.967895 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:45.968215 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:45.969073 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:45.969078 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:45.969091 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:52:45.969102 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:45.969159 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:52:45.969181 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:45.969301 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:45.969458 (+ 157us) server_negotiation.cc:299] Negotiation successful
0810 18:52:45.969648 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":148,"threads_started":1}
I20260810 18:52:45.970988 11414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88d2e05bb4af48e89d7db64018cbb3a4: Bootstrap starting.
I20260810 18:52:45.973318 19106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:52:45.973563 11414 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88d2e05bb4af48e89d7db64018cbb3a4: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:45.974246 11414 log.cc:824] T 00000000000000000000000000000000 P 88d2e05bb4af48e89d7db64018cbb3a4: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:45.976446 11414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88d2e05bb4af48e89d7db64018cbb3a4: No bootstrap required, opened a new log
I20260810 18:52:45.976919 11417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:45.975355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.129:46423 (local address 127.0.0.1:42293)
0810 18:52:45.975471 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:45.975475 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:45.975647 (+ 172us) server_negotiation.cc:407] Connection header received
0810 18:52:45.975756 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:45.975758 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:45.975783 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:45.975841 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:45.976476 (+ 635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:45.976478 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:45.976480 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:52:45.976489 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:45.976529 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:52:45.976564 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:45.976652 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:45.976786 (+ 134us) server_negotiation.cc:299] Negotiation successful
0810 18:52:45.976816 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 18:52:45.980159 11414 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88d2e05bb4af48e89d7db64018cbb3a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d2e05bb4af48e89d7db64018cbb3a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42293 } }
I20260810 18:52:45.980389 11414 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88d2e05bb4af48e89d7db64018cbb3a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:45.980438 11414 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88d2e05bb4af48e89d7db64018cbb3a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88d2e05bb4af48e89d7db64018cbb3a4, State: Initialized, Role: FOLLOWER
I20260810 18:52:45.980903 11414 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88d2e05bb4af48e89d7db64018cbb3a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d2e05bb4af48e89d7db64018cbb3a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42293 } }
I20260810 18:52:45.981026 11414 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88d2e05bb4af48e89d7db64018cbb3a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:52:45.981074 11414 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88d2e05bb4af48e89d7db64018cbb3a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:52:45.981142 11414 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88d2e05bb4af48e89d7db64018cbb3a4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:45.981992 11414 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88d2e05bb4af48e89d7db64018cbb3a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d2e05bb4af48e89d7db64018cbb3a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42293 } }
I20260810 18:52:45.982263 11414 leader_election.cc:302] T 00000000000000000000000000000000 P 88d2e05bb4af48e89d7db64018cbb3a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88d2e05bb4af48e89d7db64018cbb3a4; no voters:
I20260810 18:52:45.982487 11414 leader_election.cc:288] T 00000000000000000000000000000000 P 88d2e05bb4af48e89d7db64018cbb3a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:52:45.982616 11427 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88d2e05bb4af48e89d7db64018cbb3a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:52:45.982843 11427 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88d2e05bb4af48e89d7db64018cbb3a4 [term 1 LEADER]: Becoming Leader. State: Replica: 88d2e05bb4af48e89d7db64018cbb3a4, State: Running, Role: LEADER
I20260810 18:52:45.983129 11427 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88d2e05bb4af48e89d7db64018cbb3a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d2e05bb4af48e89d7db64018cbb3a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42293 } }
I20260810 18:52:45.983719 11414 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88d2e05bb4af48e89d7db64018cbb3a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:52:45.984870 11428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88d2e05bb4af48e89d7db64018cbb3a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88d2e05bb4af48e89d7db64018cbb3a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d2e05bb4af48e89d7db64018cbb3a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42293 } } }
I20260810 18:52:45.985000 11428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88d2e05bb4af48e89d7db64018cbb3a4 [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:45.985296 11429 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88d2e05bb4af48e89d7db64018cbb3a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88d2e05bb4af48e89d7db64018cbb3a4. Latest consensus state: current_term: 1 leader_uuid: "88d2e05bb4af48e89d7db64018cbb3a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d2e05bb4af48e89d7db64018cbb3a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42293 } } }
I20260810 18:52:45.985407 11429 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88d2e05bb4af48e89d7db64018cbb3a4 [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:45.985396 11436 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:52:45.988095 11436 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:52:45.993048 11436 catalog_manager.cc:1347] Generated new cluster ID: 48865ecdcf8246279baa90e3e717f3cc
I20260810 18:52:45.993119 11436 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:52:46.035027 11436 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:52:46.035378 11436 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:52:46.043300 11436 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88d2e05bb4af48e89d7db64018cbb3a4: Generated new TSK 0
I20260810 18:52:46.044085 11436 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:52:46.057015 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 11335
2026-08-10T18:52:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1495 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:52:46 dist-test-slave-cbsf krb5kdc[11451](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:52:46 dist-test-slave-cbsf krb5kdc[11451](info): set up 2 sockets
Aug 10 18:52:46 dist-test-slave-cbsf krb5kdc[11451](info): commencing operation
krb5kdc: starting...
W20260810 18:52:47.167987 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:52:47 dist-test-slave-cbsf krb5kdc[11451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:52:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:52:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.168.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
I20260810 18:52:47.295645 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:38977
--webserver_interface=127.18.168.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:35757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:38977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:52:47.406350 11465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:47.406622 11465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:47.406665 11465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:47.410502 11465 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:52:47.410565 11465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:47.410588 11465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:47.410606 11465 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:52:47.410625 11465 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:52:47.415903 11465 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:35757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:38977
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:38977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:47.416963 11465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:47.417968 11465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:47.426971 11470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:47.426971 11471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:47.427047 11473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:47.427524 11465 server_base.cc:1034] running on GCE node
I20260810 18:52:47.428100 11465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:47.429265 11465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:47.430490 11465 hybrid_clock.cc:648] HybridClock initialized: now 1786387967430466 us; error 49 us; skew 500 ppm
Aug 10 18:52:47 dist-test-slave-cbsf krb5kdc[11451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387967, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:47.433564 11465 init.cc:377] Logged in from keytab as kudu/127.18.168.190@KRBTEST.COM (short username kudu)
I20260810 18:52:47.434485 11465 webserver.cc:460] Webserver started at http://127.18.168.190:35621/ using document root <none> and password file <none>
I20260810 18:52:47.435006 11465 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:47.435071 11465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:47.435242 11465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:47.437362 11465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "f09411a6ca3f499d8e148d0b6ab62a6d"
format_stamp: "Formatted at 2026-08-10 18:52:47 on dist-test-slave-cbsf"
server_key: "8d255ded83fcd0aa94398cb850f5680b"
server_key_iv: "79471420ca52f3cb0a099f4197b76e71"
server_key_version: "encryptionkey@0"
I20260810 18:52:47.437911 11465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "f09411a6ca3f499d8e148d0b6ab62a6d"
format_stamp: "Formatted at 2026-08-10 18:52:47 on dist-test-slave-cbsf"
server_key: "8d255ded83fcd0aa94398cb850f5680b"
server_key_iv: "79471420ca52f3cb0a099f4197b76e71"
server_key_version: "encryptionkey@0"
I20260810 18:52:47.441789 11465 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 18:52:47.444057 11481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:47.445369 11465 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 18:52:47.445544 11465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "f09411a6ca3f499d8e148d0b6ab62a6d"
format_stamp: "Formatted at 2026-08-10 18:52:47 on dist-test-slave-cbsf"
server_key: "8d255ded83fcd0aa94398cb850f5680b"
server_key_iv: "79471420ca52f3cb0a099f4197b76e71"
server_key_version: "encryptionkey@0"
I20260810 18:52:47.445657 11465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:47.464412 11465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:47.467777 11465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:47.467981 11465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:47.475380 11465 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:38977
I20260810 18:52:47.475410 11538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:38977 every 8 connection(s)
I20260810 18:52:47.476289 11465 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:52:47.478935 11539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:47.482020 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 11465
I20260810 18:52:47.482174 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:52:47.482488 19106 external_mini_cluster.cc:1428] Setting key a70f77c7a9d6fa80be13a6927adf4221
I20260810 18:52:47.485270 11539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f09411a6ca3f499d8e148d0b6ab62a6d: Bootstrap starting.
I20260810 18:52:47.487501 11539 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f09411a6ca3f499d8e148d0b6ab62a6d: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:47.488255 11539 log.cc:824] T 00000000000000000000000000000000 P f09411a6ca3f499d8e148d0b6ab62a6d: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:47.490159 11539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f09411a6ca3f499d8e148d0b6ab62a6d: No bootstrap required, opened a new log
Aug 10 18:52:47 dist-test-slave-cbsf krb5kdc[11451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:52:47.493783 11539 raft_consensus.cc:348] T 00000000000000000000000000000000 P f09411a6ca3f499d8e148d0b6ab62a6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f09411a6ca3f499d8e148d0b6ab62a6d" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 38977 } }
I20260810 18:52:47.494040 11539 raft_consensus.cc:374] T 00000000000000000000000000000000 P f09411a6ca3f499d8e148d0b6ab62a6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:47.494130 11539 raft_consensus.cc:729] T 00000000000000000000000000000000 P f09411a6ca3f499d8e148d0b6ab62a6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f09411a6ca3f499d8e148d0b6ab62a6d, State: Initialized, Role: FOLLOWER
I20260810 18:52:47.494614 11539 consensus_queue.cc:260] T 00000000000000000000000000000000 P f09411a6ca3f499d8e148d0b6ab62a6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f09411a6ca3f499d8e148d0b6ab62a6d" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 38977 } }
I20260810 18:52:47.494774 11539 raft_consensus.cc:388] T 00000000000000000000000000000000 P f09411a6ca3f499d8e148d0b6ab62a6d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:52:47.494869 11539 raft_consensus.cc:482] T 00000000000000000000000000000000 P f09411a6ca3f499d8e148d0b6ab62a6d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:52:47.494978 11539 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f09411a6ca3f499d8e148d0b6ab62a6d [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:47.495890 11539 raft_consensus.cc:504] T 00000000000000000000000000000000 P f09411a6ca3f499d8e148d0b6ab62a6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f09411a6ca3f499d8e148d0b6ab62a6d" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 38977 } }
I20260810 18:52:47.496285 11539 leader_election.cc:302] T 00000000000000000000000000000000 P f09411a6ca3f499d8e148d0b6ab62a6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f09411a6ca3f499d8e148d0b6ab62a6d; no voters:
I20260810 18:52:47.496654 11539 leader_election.cc:288] T 00000000000000000000000000000000 P f09411a6ca3f499d8e148d0b6ab62a6d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:52:47.496748 11544 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f09411a6ca3f499d8e148d0b6ab62a6d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:52:47.496977 11544 raft_consensus.cc:686] T 00000000000000000000000000000000 P f09411a6ca3f499d8e148d0b6ab62a6d [term 1 LEADER]: Becoming Leader. State: Replica: f09411a6ca3f499d8e148d0b6ab62a6d, State: Running, Role: LEADER
I20260810 18:52:47.497118 11542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:47.483928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44729 (local address 127.18.168.190:38977)
0810 18:52:47.484376 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:47.484390 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:47.484425 (+ 35us) server_negotiation.cc:407] Connection header received
0810 18:52:47.485106 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:47.485125 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:47.485497 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:47.485823 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:47.486655 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:47.487684 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:47.488596 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:47.489132 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:47.489298 (+ 166us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:47.491878 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:47.491883 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:47.491895 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:47.491922 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:47.493687 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:47.494273 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:47.494279 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:47.494285 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:47.494404 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:47.494763 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:47.494766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:47.494768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:47.495299 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:47.495528 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:47.495879 (+ 351us) server_negotiation.cc:299] Negotiation successful
0810 18:52:47.496125 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":149,"threads_started":1}
I20260810 18:52:47.497371 11544 consensus_queue.cc:237] T 00000000000000000000000000000000 P f09411a6ca3f499d8e148d0b6ab62a6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f09411a6ca3f499d8e148d0b6ab62a6d" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 38977 } }
I20260810 18:52:47.497993 11539 sys_catalog.cc:564] T 00000000000000000000000000000000 P f09411a6ca3f499d8e148d0b6ab62a6d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:52:47.499405 11545 sys_catalog.cc:455] T 00000000000000000000000000000000 P f09411a6ca3f499d8e148d0b6ab62a6d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f09411a6ca3f499d8e148d0b6ab62a6d. Latest consensus state: current_term: 1 leader_uuid: "f09411a6ca3f499d8e148d0b6ab62a6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f09411a6ca3f499d8e148d0b6ab62a6d" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 38977 } } }
I20260810 18:52:47.499506 11545 sys_catalog.cc:458] T 00000000000000000000000000000000 P f09411a6ca3f499d8e148d0b6ab62a6d [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:47.499931 11550 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:52:47.499357 11546 sys_catalog.cc:455] T 00000000000000000000000000000000 P f09411a6ca3f499d8e148d0b6ab62a6d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f09411a6ca3f499d8e148d0b6ab62a6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f09411a6ca3f499d8e148d0b6ab62a6d" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 38977 } } }
I20260810 18:52:47.500332 11546 sys_catalog.cc:458] T 00000000000000000000000000000000 P f09411a6ca3f499d8e148d0b6ab62a6d [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:47.501349 19106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:52:47.503845 11550 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:52:47.509557 11550 catalog_manager.cc:1347] Generated new cluster ID: 0d55cbbdea194e25bc84792d48a0267b
I20260810 18:52:47.509630 11550 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:52:47.516604 11550 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:52:47.517853 11550 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:52:47.521812 11542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:47.503705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38809 (local address 127.18.168.190:38977)
0810 18:52:47.503815 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:47.503819 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:47.505772 (+ 1953us) server_negotiation.cc:407] Connection header received
0810 18:52:47.505917 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:47.505921 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:47.505976 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:47.506047 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:47.506695 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:47.507213 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:47.508297 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:47.508805 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:47.508934 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:47.513733 (+ 4799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:47.513736 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:47.513741 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:47.513769 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:47.515309 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:47.515919 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:47.515922 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:47.515924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:47.516023 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:47.520872 (+ 4849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:47.520875 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:47.520877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:47.521053 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:47.521208 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:47.521536 (+ 328us) server_negotiation.cc:299] Negotiation successful
0810 18:52:47.521632 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 18:52:47.523758 11550 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f09411a6ca3f499d8e148d0b6ab62a6d: Generated new TSK 0
I20260810 18:52:47.524477 11550 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:52:47.528232 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 11465
W20260810 18:52:47.538983 11449 connection.cc:535] client connection to 127.18.168.190:38977 recv error: Network error: failed to read from TLS socket (remote: 127.18.168.190:38977): Connection reset by peer (error 104)
2026-08-10T18:52:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1473 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:52:47 dist-test-slave-cbsf krb5kdc[11571](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:52:47 dist-test-slave-cbsf krb5kdc[11571](info): set up 2 sockets
Aug 10 18:52:47 dist-test-slave-cbsf krb5kdc[11571](info): commencing operation
krb5kdc: starting...
W20260810 18:52:48.690717 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:52:48 dist-test-slave-cbsf krb5kdc[11571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:52:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:52:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.168.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
I20260810 18:52:48.818240 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:37067
--webserver_interface=127.18.168.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:36321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:37067
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:52:48.931169 11587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:48.931417 11587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:48.931456 11587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:48.935485 11587 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:52:48.935537 11587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:48.935557 11587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:48.935575 11587 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:52:48.935590 11587 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:52:48.940667 11587 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:36321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:37067
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:37067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11587
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:48.941720 11587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:48.942703 11587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:48.951089 11594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:48.951181 11587 server_base.cc:1034] running on GCE node
W20260810 18:52:48.951097 11596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:48.951097 11593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:48.951856 11587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:48.953035 11587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:48.954210 11587 hybrid_clock.cc:648] HybridClock initialized: now 1786387968954182 us; error 39 us; skew 500 ppm
Aug 10 18:52:48 dist-test-slave-cbsf krb5kdc[11571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387968, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:48.957327 11587 init.cc:377] Logged in from keytab as kudu/127.18.168.190@KRBTEST.COM (short username kudu)
I20260810 18:52:48.958156 11587 webserver.cc:460] Webserver started at http://127.18.168.190:40863/ using document root <none> and password file <none>
I20260810 18:52:48.958683 11587 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:48.958742 11587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:48.958909 11587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:48.961025 11587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "1c2bfefec54e41cc96701128e02ad922"
format_stamp: "Formatted at 2026-08-10 18:52:48 on dist-test-slave-cbsf"
server_key: "b29e7049236482b409639ca7196861f3"
server_key_iv: "3af8ee920fbccb5cbb33381fc9a0b10e"
server_key_version: "encryptionkey@0"
I20260810 18:52:48.961560 11587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "1c2bfefec54e41cc96701128e02ad922"
format_stamp: "Formatted at 2026-08-10 18:52:48 on dist-test-slave-cbsf"
server_key: "b29e7049236482b409639ca7196861f3"
server_key_iv: "3af8ee920fbccb5cbb33381fc9a0b10e"
server_key_version: "encryptionkey@0"
I20260810 18:52:48.965221 11587 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 18:52:48.967581 11604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:48.968886 11587 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:52:48.969041 11587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "1c2bfefec54e41cc96701128e02ad922"
format_stamp: "Formatted at 2026-08-10 18:52:48 on dist-test-slave-cbsf"
server_key: "b29e7049236482b409639ca7196861f3"
server_key_iv: "3af8ee920fbccb5cbb33381fc9a0b10e"
server_key_version: "encryptionkey@0"
I20260810 18:52:48.969142 11587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:48.978976 11587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:48.981746 11587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:48.981947 11587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:48.989102 11587 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:37067
I20260810 18:52:48.989116 11659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:37067 every 8 connection(s)
I20260810 18:52:48.990090 11587 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:52:48.993407 11660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:48.995275 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 11587
I20260810 18:52:48.995559 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:52:48.996134 19106 external_mini_cluster.cc:1428] Setting key 98b45a63094ea89e2349b68d33424bd9
I20260810 18:52:49.002885 11660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c2bfefec54e41cc96701128e02ad922: Bootstrap starting.
I20260810 18:52:49.006139 11660 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c2bfefec54e41cc96701128e02ad922: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:49.007154 11660 log.cc:824] T 00000000000000000000000000000000 P 1c2bfefec54e41cc96701128e02ad922: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:49.008597 11663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:48.997814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53433 (local address 127.18.168.190:37067)
0810 18:52:48.998505 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:48.998525 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:48.998965 (+ 440us) server_negotiation.cc:407] Connection header received
0810 18:52:49.000102 (+ 1137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:49.000129 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:49.000609 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:49.000982 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:49.001942 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:49.003691 (+ 1749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:49.005093 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:49.005589 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:49.005784 (+ 195us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:49.006780 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:49.006785 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:49.006799 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:52:49.007199 (+ 400us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:49.007286 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:52:49.007322 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:49.007486 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:49.007675 (+ 189us) server_negotiation.cc:299] Negotiation successful
0810 18:52:49.007832 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":158,"threads_started":1}
I20260810 18:52:49.009660 11660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c2bfefec54e41cc96701128e02ad922: No bootstrap required, opened a new log
I20260810 18:52:49.011993 19106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:52:49.014597 11660 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c2bfefec54e41cc96701128e02ad922 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c2bfefec54e41cc96701128e02ad922" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 37067 } }
I20260810 18:52:49.014925 11660 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c2bfefec54e41cc96701128e02ad922 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:49.015064 11660 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c2bfefec54e41cc96701128e02ad922 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c2bfefec54e41cc96701128e02ad922, State: Initialized, Role: FOLLOWER
I20260810 18:52:49.015664 11660 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c2bfefec54e41cc96701128e02ad922 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c2bfefec54e41cc96701128e02ad922" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 37067 } }
I20260810 18:52:49.015877 11660 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c2bfefec54e41cc96701128e02ad922 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:52:49.015952 11660 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c2bfefec54e41cc96701128e02ad922 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:52:49.016119 11660 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c2bfefec54e41cc96701128e02ad922 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:49.017514 11660 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c2bfefec54e41cc96701128e02ad922 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c2bfefec54e41cc96701128e02ad922" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 37067 } }
I20260810 18:52:49.018079 11660 leader_election.cc:302] T 00000000000000000000000000000000 P 1c2bfefec54e41cc96701128e02ad922 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c2bfefec54e41cc96701128e02ad922; no voters:
I20260810 18:52:49.018457 11660 leader_election.cc:288] T 00000000000000000000000000000000 P 1c2bfefec54e41cc96701128e02ad922 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:52:49.019022 11670 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c2bfefec54e41cc96701128e02ad922 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:52:49.019825 11670 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c2bfefec54e41cc96701128e02ad922 [term 1 LEADER]: Becoming Leader. State: Replica: 1c2bfefec54e41cc96701128e02ad922, State: Running, Role: LEADER
I20260810 18:52:49.020076 11660 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c2bfefec54e41cc96701128e02ad922 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:52:49.020408 11663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:49.014744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53999 (local address 127.18.168.190:37067)
0810 18:52:49.014872 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:49.014876 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:49.015185 (+ 309us) server_negotiation.cc:407] Connection header received
0810 18:52:49.015369 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:49.015372 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:49.015417 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:49.015497 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:49.016026 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:49.016827 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:49.018138 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:49.018558 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:49.018662 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:49.019803 (+ 1141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:49.019806 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:49.019809 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:52:49.019823 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:49.019851 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:52:49.019865 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:49.019953 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:49.020185 (+ 232us) server_negotiation.cc:299] Negotiation successful
0810 18:52:49.020228 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 18:52:49.020790 11670 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c2bfefec54e41cc96701128e02ad922 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c2bfefec54e41cc96701128e02ad922" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 37067 } }
I20260810 18:52:49.022823 11672 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c2bfefec54e41cc96701128e02ad922 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c2bfefec54e41cc96701128e02ad922. Latest consensus state: current_term: 1 leader_uuid: "1c2bfefec54e41cc96701128e02ad922" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c2bfefec54e41cc96701128e02ad922" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 37067 } } }
I20260810 18:52:49.022804 11671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c2bfefec54e41cc96701128e02ad922 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c2bfefec54e41cc96701128e02ad922" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c2bfefec54e41cc96701128e02ad922" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 37067 } } }
I20260810 18:52:49.022970 11671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c2bfefec54e41cc96701128e02ad922 [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:49.022970 11672 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c2bfefec54e41cc96701128e02ad922 [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:49.023402 11688 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:52:49.027199 11688 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:52:49.034895 11688 catalog_manager.cc:1347] Generated new cluster ID: bf05cbbdaa364fe58aa030560f9cf711
I20260810 18:52:49.034972 11688 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:52:49.047803 11688 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:52:49.048810 11688 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:52:49.054598 11688 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c2bfefec54e41cc96701128e02ad922: Generated new TSK 0
I20260810 18:52:49.055294 11688 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:52:49.061379 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 11587
W20260810 18:52:49.071849 11569 connection.cc:535] client connection to 127.18.168.190:37067 recv error: Network error: failed to read from TLS socket (remote: 127.18.168.190:37067): Connection reset by peer (error 104)
2026-08-10T18:52:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1532 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:52:49 dist-test-slave-cbsf krb5kdc[11694](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:52:49 dist-test-slave-cbsf krb5kdc[11694](info): set up 2 sockets
Aug 10 18:52:49 dist-test-slave-cbsf krb5kdc[11694](info): commencing operation
krb5kdc: starting...
W20260810 18:52:50.180456 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:52:50 dist-test-slave-cbsf krb5kdc[11694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387970, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:52:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:52:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.168.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
I20260810 18:52:50.312222 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:41519
--webserver_interface=127.18.168.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:44581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:41519
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:52:50.424084 11709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:50.424394 11709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:50.424467 11709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:50.428529 11709 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:52:50.428608 11709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:50.428643 11709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:50.428723 11709 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:52:50.428773 11709 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:52:50.434311 11709 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:44581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:41519
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:41519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:50.435307 11709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:50.436277 11709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:50.444821 11717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:50.444880 11709 server_base.cc:1034] running on GCE node
W20260810 18:52:50.444897 11715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:50.444821 11714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:50.445649 11709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:50.446664 11709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:50.447834 11709 hybrid_clock.cc:648] HybridClock initialized: now 1786387970447817 us; error 27 us; skew 500 ppm
Aug 10 18:52:50 dist-test-slave-cbsf krb5kdc[11694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387970, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:50.455161 11709 init.cc:377] Logged in from keytab as kudu/127.18.168.190@KRBTEST.COM (short username kudu)
I20260810 18:52:50.456064 11709 webserver.cc:460] Webserver started at http://127.18.168.190:38571/ using document root <none> and password file <none>
I20260810 18:52:50.456617 11709 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:50.456678 11709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:50.456957 11709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:50.459071 11709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "17a56cde812741dc9912f4921bf34e9f"
format_stamp: "Formatted at 2026-08-10 18:52:50 on dist-test-slave-cbsf"
server_key: "4d109b546b1ac039c5a28f8aacccf0e7"
server_key_iv: "334eb2900a4bcdf8f980e9de713ee6ce"
server_key_version: "encryptionkey@0"
I20260810 18:52:50.459648 11709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "17a56cde812741dc9912f4921bf34e9f"
format_stamp: "Formatted at 2026-08-10 18:52:50 on dist-test-slave-cbsf"
server_key: "4d109b546b1ac039c5a28f8aacccf0e7"
server_key_iv: "334eb2900a4bcdf8f980e9de713ee6ce"
server_key_version: "encryptionkey@0"
I20260810 18:52:50.463330 11709 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260810 18:52:50.465663 11725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:50.466796 11709 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:52:50.466922 11709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "17a56cde812741dc9912f4921bf34e9f"
format_stamp: "Formatted at 2026-08-10 18:52:50 on dist-test-slave-cbsf"
server_key: "4d109b546b1ac039c5a28f8aacccf0e7"
server_key_iv: "334eb2900a4bcdf8f980e9de713ee6ce"
server_key_version: "encryptionkey@0"
I20260810 18:52:50.467017 11709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:50.471632 11709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:50.474989 11709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:50.475193 11709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:50.482401 11709 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:41519
I20260810 18:52:50.482435 11782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:41519 every 8 connection(s)
I20260810 18:52:50.483354 11709 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:52:50.486102 11783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:50.488483 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 11709
I20260810 18:52:50.488605 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:52:50.488933 19106 external_mini_cluster.cc:1428] Setting key 673ab17e4130ea13ef88a5a086e6dacd
I20260810 18:52:50.492017 11783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17a56cde812741dc9912f4921bf34e9f: Bootstrap starting.
I20260810 18:52:50.493981 11786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:50.490321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38061 (local address 127.18.168.190:41519)
0810 18:52:50.490760 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:50.490774 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:50.490810 (+ 36us) server_negotiation.cc:407] Connection header received
0810 18:52:50.491421 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:50.491438 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:50.491716 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:50.492044 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:50.492836 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:50.492842 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:50.492863 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:52:50.492874 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:50.492931 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:52:50.492950 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:50.493102 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:50.493172 (+ 70us) server_negotiation.cc:299] Negotiation successful
0810 18:52:50.493319 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":132,"threads_started":1}
I20260810 18:52:50.494592 11783 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17a56cde812741dc9912f4921bf34e9f: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:50.495409 11783 log.cc:824] T 00000000000000000000000000000000 P 17a56cde812741dc9912f4921bf34e9f: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:50.496776 19106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:52:50.497500 11783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17a56cde812741dc9912f4921bf34e9f: No bootstrap required, opened a new log
I20260810 18:52:50.500488 11786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:50.498918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47651 (local address 127.18.168.190:41519)
0810 18:52:50.499046 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:50.499049 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:50.499241 (+ 192us) server_negotiation.cc:407] Connection header received
0810 18:52:50.499362 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:50.499364 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:50.499388 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:50.499449 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:50.500127 (+ 678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:50.500129 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:50.500131 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:52:50.500141 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:50.500163 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:52:50.500172 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:50.500237 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:50.500341 (+ 104us) server_negotiation.cc:299] Negotiation successful
0810 18:52:50.500368 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 18:52:50.501897 11783 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17a56cde812741dc9912f4921bf34e9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17a56cde812741dc9912f4921bf34e9f" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41519 } }
I20260810 18:52:50.502146 11783 raft_consensus.cc:374] T 00000000000000000000000000000000 P 17a56cde812741dc9912f4921bf34e9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:50.502204 11783 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17a56cde812741dc9912f4921bf34e9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17a56cde812741dc9912f4921bf34e9f, State: Initialized, Role: FOLLOWER
I20260810 18:52:50.502592 11783 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17a56cde812741dc9912f4921bf34e9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17a56cde812741dc9912f4921bf34e9f" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41519 } }
I20260810 18:52:50.502748 11783 raft_consensus.cc:388] T 00000000000000000000000000000000 P 17a56cde812741dc9912f4921bf34e9f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:52:50.502838 11783 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17a56cde812741dc9912f4921bf34e9f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:52:50.502941 11783 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17a56cde812741dc9912f4921bf34e9f [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:50.503811 11783 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17a56cde812741dc9912f4921bf34e9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17a56cde812741dc9912f4921bf34e9f" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41519 } }
I20260810 18:52:50.504154 11783 leader_election.cc:302] T 00000000000000000000000000000000 P 17a56cde812741dc9912f4921bf34e9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17a56cde812741dc9912f4921bf34e9f; no voters:
I20260810 18:52:50.504448 11783 leader_election.cc:288] T 00000000000000000000000000000000 P 17a56cde812741dc9912f4921bf34e9f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:52:50.504590 11793 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17a56cde812741dc9912f4921bf34e9f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:52:50.504891 11793 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17a56cde812741dc9912f4921bf34e9f [term 1 LEADER]: Becoming Leader. State: Replica: 17a56cde812741dc9912f4921bf34e9f, State: Running, Role: LEADER
I20260810 18:52:50.505192 11793 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17a56cde812741dc9912f4921bf34e9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17a56cde812741dc9912f4921bf34e9f" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41519 } }
I20260810 18:52:50.505748 11783 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17a56cde812741dc9912f4921bf34e9f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:52:50.507259 11795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17a56cde812741dc9912f4921bf34e9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17a56cde812741dc9912f4921bf34e9f. Latest consensus state: current_term: 1 leader_uuid: "17a56cde812741dc9912f4921bf34e9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17a56cde812741dc9912f4921bf34e9f" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41519 } } }
I20260810 18:52:50.507309 11794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17a56cde812741dc9912f4921bf34e9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "17a56cde812741dc9912f4921bf34e9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17a56cde812741dc9912f4921bf34e9f" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 41519 } } }
I20260810 18:52:50.507387 11795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17a56cde812741dc9912f4921bf34e9f [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:50.507412 11794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17a56cde812741dc9912f4921bf34e9f [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:50.507776 11804 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:52:50.510825 11804 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:52:50.516919 11804 catalog_manager.cc:1347] Generated new cluster ID: 59bd6b657abc4d49a1ecd2ede50c493c
I20260810 18:52:50.516985 11804 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:52:50.557341 11804 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:52:50.557726 11804 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:52:50.566613 11804 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 17a56cde812741dc9912f4921bf34e9f: Generated new TSK 0
I20260810 18:52:50.567498 11804 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:52:50.579905 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 11709
2026-08-10T18:52:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1517 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:52:50 dist-test-slave-cbsf krb5kdc[11816](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:52:50 dist-test-slave-cbsf krb5kdc[11816](info): set up 2 sockets
Aug 10 18:52:50 dist-test-slave-cbsf krb5kdc[11816](info): commencing operation
krb5kdc: starting...
W20260810 18:52:51.699426 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:52:51 dist-test-slave-cbsf krb5kdc[11816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:52:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:52:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.168.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
I20260810 18:52:51.824225 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:44543
--webserver_interface=127.18.168.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:36467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:44543
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:52:51.934396 11830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:51.934680 11830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:51.934757 11830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:51.938668 11830 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:52:51.938756 11830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:51.938796 11830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:51.938832 11830 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:52:51.938867 11830 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:52:51.943922 11830 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:36467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:44543
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:44543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:51.944921 11830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:51.945925 11830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:51.954027 11835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:51.954025 11836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:51.954078 11838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:51.954367 11830 server_base.cc:1034] running on GCE node
I20260810 18:52:51.954857 11830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:51.955906 11830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:51.957062 11830 hybrid_clock.cc:648] HybridClock initialized: now 1786387971957044 us; error 29 us; skew 500 ppm
Aug 10 18:52:51 dist-test-slave-cbsf krb5kdc[11816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387971, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:51.959645 11830 init.cc:377] Logged in from keytab as kudu/127.18.168.190@KRBTEST.COM (short username kudu)
I20260810 18:52:51.960440 11830 webserver.cc:460] Webserver started at http://127.18.168.190:40977/ using document root <none> and password file <none>
I20260810 18:52:51.960999 11830 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:51.961067 11830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:51.961227 11830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:51.963166 11830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "863380946fcd464081c5cfd0cc515d83"
format_stamp: "Formatted at 2026-08-10 18:52:51 on dist-test-slave-cbsf"
server_key: "5b8d6e86510d7347f4aec4746040b200"
server_key_iv: "bd9a3e456031762309f73e8818e0e81b"
server_key_version: "encryptionkey@0"
I20260810 18:52:51.963661 11830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "863380946fcd464081c5cfd0cc515d83"
format_stamp: "Formatted at 2026-08-10 18:52:51 on dist-test-slave-cbsf"
server_key: "5b8d6e86510d7347f4aec4746040b200"
server_key_iv: "bd9a3e456031762309f73e8818e0e81b"
server_key_version: "encryptionkey@0"
I20260810 18:52:51.967716 11830 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:52:51.970099 11845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:51.971342 11830 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:52:51.971495 11830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "863380946fcd464081c5cfd0cc515d83"
format_stamp: "Formatted at 2026-08-10 18:52:51 on dist-test-slave-cbsf"
server_key: "5b8d6e86510d7347f4aec4746040b200"
server_key_iv: "bd9a3e456031762309f73e8818e0e81b"
server_key_version: "encryptionkey@0"
I20260810 18:52:51.971601 11830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:51.998456 11830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:52.001418 11830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:52.001626 11830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:52.008797 11830 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:44543
I20260810 18:52:52.008816 11902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:44543 every 8 connection(s)
I20260810 18:52:52.009706 11830 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:52:52.010949 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 11830
I20260810 18:52:52.011114 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:52:52.011435 19106 external_mini_cluster.cc:1428] Setting key 71a744ac7b27596dde84ee5e4a6a982a
I20260810 18:52:52.012938 11903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:52.019403 11903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 863380946fcd464081c5cfd0cc515d83: Bootstrap starting.
Aug 10 18:52:52 dist-test-slave-cbsf krb5kdc[11816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:52:52.022075 11903 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 863380946fcd464081c5cfd0cc515d83: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:52.023005 11903 log.cc:824] T 00000000000000000000000000000000 P 863380946fcd464081c5cfd0cc515d83: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:52.025308 11903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 863380946fcd464081c5cfd0cc515d83: No bootstrap required, opened a new log
I20260810 18:52:52.026149 11907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:52.012745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36133 (local address 127.18.168.190:44543)
0810 18:52:52.013170 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:52.013187 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:52.013221 (+ 34us) server_negotiation.cc:407] Connection header received
0810 18:52:52.013844 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:52.013859 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:52.014173 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:52.014447 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:52.015461 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:52.016495 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:52.017717 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:52.018177 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:52.018309 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:52.021075 (+ 2766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:52.021082 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:52.021097 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:52.021129 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:52.023098 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:52.023595 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:52.023600 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:52.023604 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:52.023711 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:52.024003 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:52.024006 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:52.024008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:52.024471 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:52.024664 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:52.024938 (+ 274us) server_negotiation.cc:299] Negotiation successful
0810 18:52:52.025226 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":131,"threads_started":1}
I20260810 18:52:52.029063 19106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:52:52.029592 11903 raft_consensus.cc:348] T 00000000000000000000000000000000 P 863380946fcd464081c5cfd0cc515d83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863380946fcd464081c5cfd0cc515d83" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 44543 } }
I20260810 18:52:52.029786 11903 raft_consensus.cc:374] T 00000000000000000000000000000000 P 863380946fcd464081c5cfd0cc515d83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:52.029850 11903 raft_consensus.cc:729] T 00000000000000000000000000000000 P 863380946fcd464081c5cfd0cc515d83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 863380946fcd464081c5cfd0cc515d83, State: Initialized, Role: FOLLOWER
I20260810 18:52:52.030284 11903 consensus_queue.cc:260] T 00000000000000000000000000000000 P 863380946fcd464081c5cfd0cc515d83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863380946fcd464081c5cfd0cc515d83" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 44543 } }
I20260810 18:52:52.030452 11903 raft_consensus.cc:388] T 00000000000000000000000000000000 P 863380946fcd464081c5cfd0cc515d83 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:52:52.030526 11903 raft_consensus.cc:482] T 00000000000000000000000000000000 P 863380946fcd464081c5cfd0cc515d83 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:52:52.030640 11903 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 863380946fcd464081c5cfd0cc515d83 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:52.031764 11903 raft_consensus.cc:504] T 00000000000000000000000000000000 P 863380946fcd464081c5cfd0cc515d83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863380946fcd464081c5cfd0cc515d83" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 44543 } }
I20260810 18:52:52.032102 11903 leader_election.cc:302] T 00000000000000000000000000000000 P 863380946fcd464081c5cfd0cc515d83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 863380946fcd464081c5cfd0cc515d83; no voters:
I20260810 18:52:52.032430 11903 leader_election.cc:288] T 00000000000000000000000000000000 P 863380946fcd464081c5cfd0cc515d83 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:52:52.032518 11914 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 863380946fcd464081c5cfd0cc515d83 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:52:52.032795 11914 raft_consensus.cc:686] T 00000000000000000000000000000000 P 863380946fcd464081c5cfd0cc515d83 [term 1 LEADER]: Becoming Leader. State: Replica: 863380946fcd464081c5cfd0cc515d83, State: Running, Role: LEADER
I20260810 18:52:52.033142 11914 consensus_queue.cc:237] T 00000000000000000000000000000000 P 863380946fcd464081c5cfd0cc515d83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863380946fcd464081c5cfd0cc515d83" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 44543 } }
I20260810 18:52:52.033818 11903 sys_catalog.cc:564] T 00000000000000000000000000000000 P 863380946fcd464081c5cfd0cc515d83 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:52:52.034320 11914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 863380946fcd464081c5cfd0cc515d83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "863380946fcd464081c5cfd0cc515d83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863380946fcd464081c5cfd0cc515d83" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 44543 } } }
I20260810 18:52:52.034493 11914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 863380946fcd464081c5cfd0cc515d83 [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:52.034729 11915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 863380946fcd464081c5cfd0cc515d83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 863380946fcd464081c5cfd0cc515d83. Latest consensus state: current_term: 1 leader_uuid: "863380946fcd464081c5cfd0cc515d83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863380946fcd464081c5cfd0cc515d83" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 44543 } } }
I20260810 18:52:52.034835 11915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 863380946fcd464081c5cfd0cc515d83 [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:52.034950 11921 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:52:52.038712 11921 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:52:52.040479 11907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:52.031251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59111 (local address 127.18.168.190:44543)
0810 18:52:52.031346 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:52.031350 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:52.031628 (+ 278us) server_negotiation.cc:407] Connection header received
0810 18:52:52.031695 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:52.031701 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:52.031755 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:52.031842 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:52.032672 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:52.033445 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:52.034455 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:52.034869 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:52.034951 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:52.036731 (+ 1780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:52.036735 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:52.036737 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:52.036763 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:52.038354 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:52.038879 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:52.038885 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:52.038889 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:52.038980 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:52.039580 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:52.039584 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:52.039586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:52.039776 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:52.039910 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:52.040182 (+ 272us) server_negotiation.cc:299] Negotiation successful
0810 18:52:52.040298 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 18:52:52.045692 11921 catalog_manager.cc:1347] Generated new cluster ID: b39c4d97726d474db6a99bdda37bb6c8
I20260810 18:52:52.045768 11921 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:52:52.063374 11921 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:52:52.064370 11921 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:52:52.069832 11921 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 863380946fcd464081c5cfd0cc515d83: Generated new TSK 0
I20260810 18:52:52.070497 11921 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:52:52.074860 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 11830
2026-08-10T18:52:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1494 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:52:52 dist-test-slave-cbsf krb5kdc[11936](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:52:52 dist-test-slave-cbsf krb5kdc[11936](info): set up 2 sockets
Aug 10 18:52:52 dist-test-slave-cbsf krb5kdc[11936](info): commencing operation
krb5kdc: starting...
W20260810 18:52:53.192346 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:52:53 dist-test-slave-cbsf krb5kdc[11936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:52:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:52:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.168.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
I20260810 18:52:53.321777 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:37191
--webserver_interface=127.18.168.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:45301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:37191
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:52:53.432140 11950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:53.432375 11950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:53.432413 11950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:53.436209 11950 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:52:53.436262 11950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:53.436281 11950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:53.436296 11950 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:52:53.436311 11950 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:52:53.441354 11950 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:45301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:37191
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:37191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11950
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:53.442265 11950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:53.443168 11950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:53.451427 11958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:53.451457 11956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:53.451644 11950 server_base.cc:1034] running on GCE node
W20260810 18:52:53.451457 11955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:53.452229 11950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:53.453418 11950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:53.454610 11950 hybrid_clock.cc:648] HybridClock initialized: now 1786387973454591 us; error 29 us; skew 500 ppm
Aug 10 18:52:53 dist-test-slave-cbsf krb5kdc[11936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387973, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:53.457504 11950 init.cc:377] Logged in from keytab as kudu/127.18.168.190@KRBTEST.COM (short username kudu)
I20260810 18:52:53.458367 11950 webserver.cc:460] Webserver started at http://127.18.168.190:40523/ using document root <none> and password file <none>
I20260810 18:52:53.458921 11950 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:53.459012 11950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:53.459283 11950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:53.461371 11950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "128f2ead7f5a4c9783e1b9ee87354cbd"
format_stamp: "Formatted at 2026-08-10 18:52:53 on dist-test-slave-cbsf"
server_key: "8a36eec173ec0946163d305f4d0d2ddd"
server_key_iv: "7bf9b41f1289b0ec789709e9b0aac362"
server_key_version: "encryptionkey@0"
I20260810 18:52:53.461930 11950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "128f2ead7f5a4c9783e1b9ee87354cbd"
format_stamp: "Formatted at 2026-08-10 18:52:53 on dist-test-slave-cbsf"
server_key: "8a36eec173ec0946163d305f4d0d2ddd"
server_key_iv: "7bf9b41f1289b0ec789709e9b0aac362"
server_key_version: "encryptionkey@0"
I20260810 18:52:53.465637 11950 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 18:52:53.467895 11965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:53.469168 11950 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260810 18:52:53.469311 11950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "128f2ead7f5a4c9783e1b9ee87354cbd"
format_stamp: "Formatted at 2026-08-10 18:52:53 on dist-test-slave-cbsf"
server_key: "8a36eec173ec0946163d305f4d0d2ddd"
server_key_iv: "7bf9b41f1289b0ec789709e9b0aac362"
server_key_version: "encryptionkey@0"
I20260810 18:52:53.469419 11950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:53.503393 11950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:53.506767 11950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:53.506994 11950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:53.514169 11950 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:37191
I20260810 18:52:53.514196 12021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:37191 every 8 connection(s)
I20260810 18:52:53.515100 11950 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:52:53.517786 12023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:53.519073 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 11950
I20260810 18:52:53.519222 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:52:53.519554 19106 external_mini_cluster.cc:1428] Setting key a01cc4eb59c6236c3c171a75672707f7
I20260810 18:52:53.523824 12023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 128f2ead7f5a4c9783e1b9ee87354cbd: Bootstrap starting.
I20260810 18:52:53.526057 12023 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 128f2ead7f5a4c9783e1b9ee87354cbd: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:53.526746 12023 log.cc:824] T 00000000000000000000000000000000 P 128f2ead7f5a4c9783e1b9ee87354cbd: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:53.528605 12026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:53.521021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55039 (local address 127.18.168.190:37191)
0810 18:52:53.521465 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:53.521479 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:53.521517 (+ 38us) server_negotiation.cc:407] Connection header received
0810 18:52:53.522158 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:53.522174 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:53.522483 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:53.522797 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:53.523243 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:53.524248 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:53.525270 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:53.525741 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:53.525907 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:53.526743 (+ 836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:53.526748 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:53.526761 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:52:53.527246 (+ 485us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:53.527313 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:52:53.527347 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:53.527453 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:53.527632 (+ 179us) server_negotiation.cc:299] Negotiation successful
0810 18:52:53.527840 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":133,"threads_started":1}
I20260810 18:52:53.528699 12023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 128f2ead7f5a4c9783e1b9ee87354cbd: No bootstrap required, opened a new log
I20260810 18:52:53.531379 19106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:52:53.533041 12023 raft_consensus.cc:348] T 00000000000000000000000000000000 P 128f2ead7f5a4c9783e1b9ee87354cbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "128f2ead7f5a4c9783e1b9ee87354cbd" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 37191 } }
I20260810 18:52:53.533335 12023 raft_consensus.cc:374] T 00000000000000000000000000000000 P 128f2ead7f5a4c9783e1b9ee87354cbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:53.533432 12023 raft_consensus.cc:729] T 00000000000000000000000000000000 P 128f2ead7f5a4c9783e1b9ee87354cbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 128f2ead7f5a4c9783e1b9ee87354cbd, State: Initialized, Role: FOLLOWER
I20260810 18:52:53.533926 12023 consensus_queue.cc:260] T 00000000000000000000000000000000 P 128f2ead7f5a4c9783e1b9ee87354cbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "128f2ead7f5a4c9783e1b9ee87354cbd" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 37191 } }
I20260810 18:52:53.534118 12023 raft_consensus.cc:388] T 00000000000000000000000000000000 P 128f2ead7f5a4c9783e1b9ee87354cbd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:52:53.534242 12023 raft_consensus.cc:482] T 00000000000000000000000000000000 P 128f2ead7f5a4c9783e1b9ee87354cbd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:52:53.534374 12023 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 128f2ead7f5a4c9783e1b9ee87354cbd [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:53.535228 12023 raft_consensus.cc:504] T 00000000000000000000000000000000 P 128f2ead7f5a4c9783e1b9ee87354cbd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "128f2ead7f5a4c9783e1b9ee87354cbd" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 37191 } }
I20260810 18:52:53.535561 12023 leader_election.cc:302] T 00000000000000000000000000000000 P 128f2ead7f5a4c9783e1b9ee87354cbd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 128f2ead7f5a4c9783e1b9ee87354cbd; no voters:
I20260810 18:52:53.535861 12023 leader_election.cc:288] T 00000000000000000000000000000000 P 128f2ead7f5a4c9783e1b9ee87354cbd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:52:53.535979 12033 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 128f2ead7f5a4c9783e1b9ee87354cbd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:52:53.536247 12033 raft_consensus.cc:686] T 00000000000000000000000000000000 P 128f2ead7f5a4c9783e1b9ee87354cbd [term 1 LEADER]: Becoming Leader. State: Replica: 128f2ead7f5a4c9783e1b9ee87354cbd, State: Running, Role: LEADER
I20260810 18:52:53.536600 12033 consensus_queue.cc:237] T 00000000000000000000000000000000 P 128f2ead7f5a4c9783e1b9ee87354cbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "128f2ead7f5a4c9783e1b9ee87354cbd" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 37191 } }
I20260810 18:52:53.537029 12023 sys_catalog.cc:564] T 00000000000000000000000000000000 P 128f2ead7f5a4c9783e1b9ee87354cbd [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:52:53.538309 12026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:53.533740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45769 (local address 127.18.168.190:37191)
0810 18:52:53.533865 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:53.533870 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:53.534087 (+ 217us) server_negotiation.cc:407] Connection header received
0810 18:52:53.534229 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:53.534234 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:53.534282 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:53.534370 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:53.534650 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:53.535146 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:53.536219 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:53.536557 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:53.536638 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:53.537650 (+ 1012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:53.537654 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:53.537656 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:52:53.537668 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:53.537696 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:52:53.537708 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:53.537793 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:53.538118 (+ 325us) server_negotiation.cc:299] Negotiation successful
0810 18:52:53.538176 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 18:52:53.538718 12036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 128f2ead7f5a4c9783e1b9ee87354cbd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 128f2ead7f5a4c9783e1b9ee87354cbd. Latest consensus state: current_term: 1 leader_uuid: "128f2ead7f5a4c9783e1b9ee87354cbd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "128f2ead7f5a4c9783e1b9ee87354cbd" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 37191 } } }
I20260810 18:52:53.538843 12036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 128f2ead7f5a4c9783e1b9ee87354cbd [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:53.539150 12035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 128f2ead7f5a4c9783e1b9ee87354cbd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "128f2ead7f5a4c9783e1b9ee87354cbd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "128f2ead7f5a4c9783e1b9ee87354cbd" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 37191 } } }
I20260810 18:52:53.539243 12035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 128f2ead7f5a4c9783e1b9ee87354cbd [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:53.539621 12051 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:52:53.542590 12051 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:52:53.548027 12051 catalog_manager.cc:1347] Generated new cluster ID: 0b65b3e6d21a4ef6a0ee3b4039e44f08
I20260810 18:52:53.548089 12051 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:52:53.557821 12051 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:52:53.558701 12051 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:52:53.565618 12051 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 128f2ead7f5a4c9783e1b9ee87354cbd: Generated new TSK 0
I20260810 18:52:53.566234 12051 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:52:53.575958 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 11950
2026-08-10T18:52:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1502 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:52:53 dist-test-slave-cbsf krb5kdc[12058](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:52:53 dist-test-slave-cbsf krb5kdc[12058](info): set up 2 sockets
Aug 10 18:52:53 dist-test-slave-cbsf krb5kdc[12058](info): commencing operation
krb5kdc: starting...
W20260810 18:52:54.747505 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:52:54 dist-test-slave-cbsf krb5kdc[12058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:52:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:52:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.168.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
I20260810 18:52:54.875494 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:42813
--webserver_interface=127.18.168.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:41291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:42813
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:52:54.985044 12072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:54.985486 12072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:54.985556 12072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:54.989423 12072 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:52:54.989480 12072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:54.989498 12072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:54.989514 12072 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:52:54.989528 12072 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:52:54.997951 12072 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:41291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:42813
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:42813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:54.998951 12072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:54.999958 12072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:55.008440 12080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:55.008477 12078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:55.008517 12077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:55.008589 12072 server_base.cc:1034] running on GCE node
I20260810 18:52:55.009302 12072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:55.010354 12072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:55.011548 12072 hybrid_clock.cc:648] HybridClock initialized: now 1786387975011526 us; error 44 us; skew 500 ppm
Aug 10 18:52:55 dist-test-slave-cbsf krb5kdc[12058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387975, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:55.014313 12072 init.cc:377] Logged in from keytab as kudu/127.18.168.190@KRBTEST.COM (short username kudu)
I20260810 18:52:55.015214 12072 webserver.cc:460] Webserver started at http://127.18.168.190:40419/ using document root <none> and password file <none>
I20260810 18:52:55.015755 12072 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:55.015862 12072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:55.016109 12072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:55.018167 12072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "958c0553f5ce4ba3929efe9b80b314bf"
format_stamp: "Formatted at 2026-08-10 18:52:55 on dist-test-slave-cbsf"
server_key: "097f3a1030e569f04c51c80ddd7e2821"
server_key_iv: "28bcd70a313a5b0c11b8e4a0d6123fd2"
server_key_version: "encryptionkey@0"
I20260810 18:52:55.018733 12072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "958c0553f5ce4ba3929efe9b80b314bf"
format_stamp: "Formatted at 2026-08-10 18:52:55 on dist-test-slave-cbsf"
server_key: "097f3a1030e569f04c51c80ddd7e2821"
server_key_iv: "28bcd70a313a5b0c11b8e4a0d6123fd2"
server_key_version: "encryptionkey@0"
I20260810 18:52:55.022495 12072 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 18:52:55.024911 12088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:55.026175 12072 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:52:55.026319 12072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "958c0553f5ce4ba3929efe9b80b314bf"
format_stamp: "Formatted at 2026-08-10 18:52:55 on dist-test-slave-cbsf"
server_key: "097f3a1030e569f04c51c80ddd7e2821"
server_key_iv: "28bcd70a313a5b0c11b8e4a0d6123fd2"
server_key_version: "encryptionkey@0"
I20260810 18:52:55.026427 12072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:55.031028 12072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:55.034140 12072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:55.034339 12072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:55.041729 12147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:42813 every 8 connection(s)
I20260810 18:52:55.041734 12072 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:42813
I20260810 18:52:55.042644 12072 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:52:55.045442 12148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:55.050824 12148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 958c0553f5ce4ba3929efe9b80b314bf: Bootstrap starting.
I20260810 18:52:55.051911 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 12072
I20260810 18:52:55.052057 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:52:55.052385 19106 external_mini_cluster.cc:1428] Setting key 2355103a1acf43da667be227f754020b
I20260810 18:52:55.053849 12148 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 958c0553f5ce4ba3929efe9b80b314bf: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:55.054903 12148 log.cc:824] T 00000000000000000000000000000000 P 958c0553f5ce4ba3929efe9b80b314bf: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:55.057170 12148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 958c0553f5ce4ba3929efe9b80b314bf: No bootstrap required, opened a new log
I20260810 18:52:55.058835 12151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:55.053728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35909 (local address 127.18.168.190:42813)
0810 18:52:55.054311 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:55.054330 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:55.054376 (+ 46us) server_negotiation.cc:407] Connection header received
0810 18:52:55.056052 (+ 1676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:55.056073 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:55.056352 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:55.056697 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:55.057584 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:55.057589 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:55.057610 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:52:55.057621 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:55.057676 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:52:55.057695 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:55.057816 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:55.057915 (+ 99us) server_negotiation.cc:299] Negotiation successful
0810 18:52:55.058091 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":139,"threads_started":1}
I20260810 18:52:55.060873 12148 raft_consensus.cc:348] T 00000000000000000000000000000000 P 958c0553f5ce4ba3929efe9b80b314bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "958c0553f5ce4ba3929efe9b80b314bf" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 42813 } }
I20260810 18:52:55.061177 12148 raft_consensus.cc:374] T 00000000000000000000000000000000 P 958c0553f5ce4ba3929efe9b80b314bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:55.061244 12148 raft_consensus.cc:729] T 00000000000000000000000000000000 P 958c0553f5ce4ba3929efe9b80b314bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 958c0553f5ce4ba3929efe9b80b314bf, State: Initialized, Role: FOLLOWER
I20260810 18:52:55.061728 12148 consensus_queue.cc:260] T 00000000000000000000000000000000 P 958c0553f5ce4ba3929efe9b80b314bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "958c0553f5ce4ba3929efe9b80b314bf" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 42813 } }
I20260810 18:52:55.061885 12148 raft_consensus.cc:388] T 00000000000000000000000000000000 P 958c0553f5ce4ba3929efe9b80b314bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:52:55.061993 12148 raft_consensus.cc:482] T 00000000000000000000000000000000 P 958c0553f5ce4ba3929efe9b80b314bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:52:55.062131 12148 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 958c0553f5ce4ba3929efe9b80b314bf [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:55.062156 19106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:52:55.063097 12148 raft_consensus.cc:504] T 00000000000000000000000000000000 P 958c0553f5ce4ba3929efe9b80b314bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "958c0553f5ce4ba3929efe9b80b314bf" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 42813 } }
I20260810 18:52:55.063474 12148 leader_election.cc:302] T 00000000000000000000000000000000 P 958c0553f5ce4ba3929efe9b80b314bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 958c0553f5ce4ba3929efe9b80b314bf; no voters:
I20260810 18:52:55.063839 12148 leader_election.cc:288] T 00000000000000000000000000000000 P 958c0553f5ce4ba3929efe9b80b314bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:52:55.063953 12158 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 958c0553f5ce4ba3929efe9b80b314bf [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:52:55.064179 12158 raft_consensus.cc:686] T 00000000000000000000000000000000 P 958c0553f5ce4ba3929efe9b80b314bf [term 1 LEADER]: Becoming Leader. State: Replica: 958c0553f5ce4ba3929efe9b80b314bf, State: Running, Role: LEADER
I20260810 18:52:55.064673 12158 consensus_queue.cc:237] T 00000000000000000000000000000000 P 958c0553f5ce4ba3929efe9b80b314bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "958c0553f5ce4ba3929efe9b80b314bf" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 42813 } }
I20260810 18:52:55.065361 12148 sys_catalog.cc:564] T 00000000000000000000000000000000 P 958c0553f5ce4ba3929efe9b80b314bf [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:52:55.065951 12158 sys_catalog.cc:455] T 00000000000000000000000000000000 P 958c0553f5ce4ba3929efe9b80b314bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 958c0553f5ce4ba3929efe9b80b314bf. Latest consensus state: current_term: 1 leader_uuid: "958c0553f5ce4ba3929efe9b80b314bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "958c0553f5ce4ba3929efe9b80b314bf" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 42813 } } }
I20260810 18:52:55.066120 12158 sys_catalog.cc:458] T 00000000000000000000000000000000 P 958c0553f5ce4ba3929efe9b80b314bf [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:55.066608 12159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 958c0553f5ce4ba3929efe9b80b314bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "958c0553f5ce4ba3929efe9b80b314bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "958c0553f5ce4ba3929efe9b80b314bf" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 42813 } } }
I20260810 18:52:55.066722 12159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 958c0553f5ce4ba3929efe9b80b314bf [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:55.066748 12151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:55.064363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44827 (local address 127.18.168.190:42813)
0810 18:52:55.064456 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:55.064462 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:55.064913 (+ 451us) server_negotiation.cc:407] Connection header received
0810 18:52:55.065055 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:55.065057 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:55.065093 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:55.065173 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:55.066059 (+ 886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:55.066062 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:55.066065 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:52:55.066076 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:55.066104 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:52:55.066118 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:55.066194 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:55.066515 (+ 321us) server_negotiation.cc:299] Negotiation successful
0810 18:52:55.066558 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260810 18:52:55.070276 12177 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 958c0553f5ce4ba3929efe9b80b314bf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:52:55.070343 12177 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:52:55.070429 12164 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:52:55.071102 12164 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:52:55.076870 12164 catalog_manager.cc:1347] Generated new cluster ID: 2a667ee5652e4c8f928368d69ae1d3b1
I20260810 18:52:55.076941 12164 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:52:55.091282 12164 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:52:55.091626 12164 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:52:55.095834 12164 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 958c0553f5ce4ba3929efe9b80b314bf: Generated new TSK 0
I20260810 18:52:55.096583 12164 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:52:55.103447 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 12072
2026-08-10T18:52:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1526 ms)
[----------] 18 tests from AuthTokenIssuingTest (26923 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:52:55 dist-test-slave-cbsf krb5kdc[12183](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:52:55 dist-test-slave-cbsf krb5kdc[12183](info): set up 2 sockets
Aug 10 18:52:55 dist-test-slave-cbsf krb5kdc[12183](info): commencing operation
krb5kdc: starting...
W20260810 18:52:56.215305 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:52:56 dist-test-slave-cbsf krb5kdc[12183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387976, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:52:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:52:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.168.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
I20260810 18:52:56.337847 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:43139
--webserver_interface=127.18.168.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:40257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:43139,127.18.168.189:33023,127.18.168.188:46205
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:52:56.447047 12197 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 18:52:56.447408 12197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:56.447477 12197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:56.447551 12197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:56.451414 12197 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:52:56.451493 12197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:56.451529 12197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:56.451563 12197 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:52:56.451599 12197 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:52:56.456784 12197 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:40257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:43139,127.18.168.189:33023,127.18.168.188:46205
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:43139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:56.457793 12197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:56.458775 12197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:56.467499 12203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:56.467489 12206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:56.467782 12197 server_base.cc:1034] running on GCE node
W20260810 18:52:56.467469 12204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:56.468307 12197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:56.469399 12197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:56.470602 12197 hybrid_clock.cc:648] HybridClock initialized: now 1786387976470576 us; error 45 us; skew 500 ppm
Aug 10 18:52:56 dist-test-slave-cbsf krb5kdc[12183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387976, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:56.478382 12197 init.cc:377] Logged in from keytab as kudu/127.18.168.190@KRBTEST.COM (short username kudu)
I20260810 18:52:56.479244 12197 webserver.cc:460] Webserver started at http://127.18.168.190:34913/ using document root <none> and password file <none>
I20260810 18:52:56.479818 12197 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:56.479910 12197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:56.480154 12197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:56.482262 12197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "980384115e344d769dce6dd222d03a8c"
format_stamp: "Formatted at 2026-08-10 18:52:56 on dist-test-slave-cbsf"
server_key: "09ecd092f825447521034bf3f87d53fc"
server_key_iv: "26f1d726001a1099132b9876df0e72ef"
server_key_version: "encryptionkey@0"
I20260810 18:52:56.482828 12197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "980384115e344d769dce6dd222d03a8c"
format_stamp: "Formatted at 2026-08-10 18:52:56 on dist-test-slave-cbsf"
server_key: "09ecd092f825447521034bf3f87d53fc"
server_key_iv: "26f1d726001a1099132b9876df0e72ef"
server_key_version: "encryptionkey@0"
I20260810 18:52:56.486598 12197 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260810 18:52:56.489101 12214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:56.490302 12197 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 18:52:56.490444 12197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "980384115e344d769dce6dd222d03a8c"
format_stamp: "Formatted at 2026-08-10 18:52:56 on dist-test-slave-cbsf"
server_key: "09ecd092f825447521034bf3f87d53fc"
server_key_iv: "26f1d726001a1099132b9876df0e72ef"
server_key_version: "encryptionkey@0"
I20260810 18:52:56.490551 12197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:56.508886 12197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:56.512158 12197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:56.512357 12197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:56.519508 12197 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:43139
I20260810 18:52:56.519793 12272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:43139 every 8 connection(s)
I20260810 18:52:56.520474 12197 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:52:56.523169 12273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:56.524256 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 12197
I20260810 18:52:56.524394 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:52:56.524681 19106 external_mini_cluster.cc:1428] Setting key 23c6fab8d20f6e5f0b2961d9d25779d6
I20260810 18:52:56.527077 12273 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43139 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.168.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.189@KRBTEST.COM" created.
Aug 10 18:52:56 dist-test-slave-cbsf krb5kdc[12183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387976, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:52:56.543294 12277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.529726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42418 (local address 127.18.168.190:43139)
0810 18:52:56.530297 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:56.530312 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:56.530375 (+ 63us) server_negotiation.cc:407] Connection header received
0810 18:52:56.531470 (+ 1095us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:56.531478 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:56.531747 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:56.531982 (+ 235us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:56.532773 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:56.534250 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:56.535285 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:56.535668 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:56.535834 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:56.538577 (+ 2743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:56.538587 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:56.538603 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:56.538633 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:56.540327 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:56.540950 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:56.540955 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:56.540956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:56.541052 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:56.541411 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:56.541413 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:56.541414 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:56.541753 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:56.541852 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:56.542343 (+ 491us) server_negotiation.cc:299] Negotiation successful
0810 18:52:56.542524 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":154,"threads_started":1}
I20260810 18:52:56.543299 12276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.529725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43139 (local address 127.0.0.1:42418)
0810 18:52:56.530194 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:56.530245 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:56.531133 (+ 888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:56.531974 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:56.531978 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:56.532323 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:52:56.532593 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:56.532607 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:56.534378 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:56.534381 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:56.535170 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:56.535180 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:56.535879 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:56.535886 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:56.536002 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:56.536513 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:52:56.536529 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:52:56.538421 (+ 1892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:52:56.540444 (+ 2023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:56.540457 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:56.540469 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:56.540847 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:56.541151 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:56.541157 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:56.541162 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:56.541316 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:56.541881 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:52:56.541891 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:52:56.542075 (+ 184us) client_negotiation.cc:769] Sending connection context
0810 18:52:56.542280 (+ 205us) client_negotiation.cc:240] Negotiation successful
0810 18:52:56.542662 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":51,"thread_start_us":132,"threads_started":1}
I20260810 18:52:56.545697 12273 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33023 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:52:56.546397 12276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.546170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.189:33023 (local address 127.0.0.1:46780)
0810 18:52:56.546281 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:56.546345 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.189:33023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 18:52:56.546569 12215 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.168.189:33023: connect: Connection refused (error 111)
I20260810 18:52:56.547417 12276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.547207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.189:33023 (local address 127.0.0.1:46790)
0810 18:52:56.547320 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:56.547370 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.189:33023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 18:52:56.547549 12273 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.168.189:33023: Network error: Client connection negotiation failed: client connection to 127.18.168.189:33023: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.189.keytab.
Entry for principal kudu/127.18.168.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.189.keytab.
I20260810 18:52:56.553697 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.189:33023
--webserver_interface=127.18.168.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.189.keytab
--principal=kudu/127.18.168.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:40257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:43139,127.18.168.189:33023,127.18.168.188:46205
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:52:56.596825 12273 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33023 } attempt: 1
I20260810 18:52:56.597697 12276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.597391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.189:33023 (local address 127.0.0.1:46796)
0810 18:52:56.597548 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:56.597619 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.189:33023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260810 18:52:56.598467 12276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.598261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.189:33023 (local address 127.0.0.1:46806)
0810 18:52:56.598366 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:56.598411 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.189:33023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260810 18:52:56.598616 12273 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.168.189:33023: Network error: Client connection negotiation failed: client connection to 127.18.168.189:33023: connect: Connection refused (error 111)
W20260810 18:52:56.664400 12281 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 18:52:56.664649 12281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:56.664707 12281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:56.664799 12281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260810 18:52:56.666786 12273 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33023 } attempt: 2
I20260810 18:52:56.667608 12276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.667320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.189:33023 (local address 127.0.0.1:46820)
0810 18:52:56.667472 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:56.667532 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.189:33023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260810 18:52:56.668277 12276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.668094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.189:33023 (local address 127.0.0.1:46826)
0810 18:52:56.668185 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:56.668226 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.189:33023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260810 18:52:56.668411 12273 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.168.189:33023: Network error: Client connection negotiation failed: client connection to 127.18.168.189:33023: connect: Connection refused (error 111)
W20260810 18:52:56.669276 12281 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:52:56.669327 12281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:56.669346 12281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:56.669363 12281 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:52:56.669384 12281 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:52:56.674486 12281 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:40257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:43139,127.18.168.189:33023,127.18.168.188:46205
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.189.keytab
--principal=kudu/127.18.168.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.189:33023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.168.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:56.675398 12281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:56.676313 12281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:56.684337 12287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:56.684392 12289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:56.684335 12291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:56.684576 12281 server_base.cc:1034] running on GCE node
I20260810 18:52:56.685245 12281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:56.686305 12281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:56.687492 12281 hybrid_clock.cc:648] HybridClock initialized: now 1786387976687464 us; error 50 us; skew 500 ppm
Aug 10 18:52:56 dist-test-slave-cbsf krb5kdc[12183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387976, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:56.690376 12281 init.cc:377] Logged in from keytab as kudu/127.18.168.189@KRBTEST.COM (short username kudu)
I20260810 18:52:56.691303 12281 webserver.cc:460] Webserver started at http://127.18.168.189:36503/ using document root <none> and password file <none>
I20260810 18:52:56.691852 12281 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:56.691944 12281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:56.692175 12281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:56.694289 12281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-1/data/instance:
uuid: "90a8c9bc1fea49cfb4ca1ed1e9eff9c6"
format_stamp: "Formatted at 2026-08-10 18:52:56 on dist-test-slave-cbsf"
server_key: "a8b44eaa976ffe88d57d615ccaafb433"
server_key_iv: "11aa7d2e36371957b0ec0cd6a4e4644f"
server_key_version: "encryptionkey@0"
I20260810 18:52:56.694903 12281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-1/wal/instance:
uuid: "90a8c9bc1fea49cfb4ca1ed1e9eff9c6"
format_stamp: "Formatted at 2026-08-10 18:52:56 on dist-test-slave-cbsf"
server_key: "a8b44eaa976ffe88d57d615ccaafb433"
server_key_iv: "11aa7d2e36371957b0ec0cd6a4e4644f"
server_key_version: "encryptionkey@0"
I20260810 18:52:56.698740 12281 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260810 18:52:56.701239 12298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:56.702400 12281 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:52:56.702543 12281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-1/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-1/wal
uuid: "90a8c9bc1fea49cfb4ca1ed1e9eff9c6"
format_stamp: "Formatted at 2026-08-10 18:52:56 on dist-test-slave-cbsf"
server_key: "a8b44eaa976ffe88d57d615ccaafb433"
server_key_iv: "11aa7d2e36371957b0ec0cd6a4e4644f"
server_key_version: "encryptionkey@0"
I20260810 18:52:56.702658 12281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:56.721246 12281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:56.724417 12281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:56.724635 12281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:56.731876 12281 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.189:33023
I20260810 18:52:56.732167 12359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.189:33023 every 8 connection(s)
I20260810 18:52:56.732904 12281 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-1/data/info.pb
I20260810 18:52:56.735507 12360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:56.739161 12360 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43139 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:52:56.740365 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 12281
I20260810 18:52:56.740475 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-1/wal/instance
I20260810 18:52:56.740803 19106 external_mini_cluster.cc:1428] Setting key 829e6480bd45d4a2ff574b76e0859e19
Aug 10 18:52:56 dist-test-slave-cbsf krb5kdc[12183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387976, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.189@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.168.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.188@KRBTEST.COM" created.
I20260810 18:52:56.753324 12277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.741053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42432 (local address 127.18.168.190:43139)
0810 18:52:56.741241 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:56.741245 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:56.742097 (+ 852us) server_negotiation.cc:407] Connection header received
0810 18:52:56.743207 (+ 1110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:56.743210 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:56.743259 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:56.743330 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:56.744927 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:56.745442 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:56.746529 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:56.746768 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:56.746887 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:56.749615 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:56.749618 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:56.749620 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:56.749643 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:56.751029 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:56.751548 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:56.751552 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:56.751554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:56.751647 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:56.751990 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:56.751995 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:56.751997 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:56.752220 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:56.752357 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:56.752984 (+ 627us) server_negotiation.cc:299] Negotiation successful
0810 18:52:56.753140 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260810 18:52:56.754184 12362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.741445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43139 (local address 127.0.0.1:42432)
0810 18:52:56.741948 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:56.741989 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:56.742955 (+ 966us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:56.743569 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:56.743580 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:56.744064 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:52:56.744770 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:56.744794 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:56.745571 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:56.745574 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:56.746401 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:56.746411 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:56.746925 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:56.746929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:56.747107 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:56.747670 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:52:56.747685 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:52:56.749407 (+ 1722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:52:56.751164 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:56.751170 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:56.751183 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:56.751439 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:56.751742 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:56.751746 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:56.751748 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:56.751892 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:56.752358 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:52:56.752365 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:52:56.752611 (+ 246us) client_negotiation.cc:769] Sending connection context
0810 18:52:56.752913 (+ 302us) client_negotiation.cc:240] Negotiation successful
0810 18:52:56.753275 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":154,"threads_started":1}
I20260810 18:52:56.755647 12360 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33023 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:52:56.759582 12273 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33023 } attempt: 3
Aug 10 18:52:56 dist-test-slave-cbsf krb5kdc[12183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387976, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.189@KRBTEST.COM for kudu/127.18.168.189@KRBTEST.COM
I20260810 18:52:56.765537 12362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.756135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.189:33023 (local address 127.0.0.1:46842)
0810 18:52:56.756250 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:56.756264 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:56.756336 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:56.757030 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:56.757033 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:56.757220 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:52:56.757291 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:56.757297 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:56.758234 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:56.758240 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:56.758755 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:56.758763 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:56.759356 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:56.759359 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:56.759496 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:56.759983 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:52:56.760002 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:52:56.761555 (+ 1553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:52:56.763840 (+ 2285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:56.763843 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:56.763844 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:56.764141 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:56.764457 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:56.764463 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:56.764468 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:56.764567 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:56.765177 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:52:56.765179 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:52:56.765256 (+ 77us) client_negotiation.cc:769] Sending connection context
0810 18:52:56.765339 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 18:52:56.765429 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":20}
I20260810 18:52:56.766065 12366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.756250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46842 (local address 127.18.168.189:33023)
0810 18:52:56.756667 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:56.756679 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:56.756724 (+ 45us) server_negotiation.cc:407] Connection header received
0810 18:52:56.756762 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:56.756767 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:56.756914 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:56.757050 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:56.757397 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:56.758117 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:56.758868 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:56.759243 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:56.759359 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:56.761693 (+ 2334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:56.761707 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:56.761724 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:56.761746 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:56.763724 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:56.764277 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:56.764282 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:56.764284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:56.764365 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:56.764688 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:56.764691 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:56.764693 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:56.765065 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:56.765206 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:56.765449 (+ 243us) server_negotiation.cc:299] Negotiation successful
0810 18:52:56.765629 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":274,"thread_start_us":155,"threads_started":1}
Aug 10 18:52:56 dist-test-slave-cbsf krb5kdc[12183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387976, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for kudu/127.18.168.189@KRBTEST.COM
I20260810 18:52:56.767966 12360 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46205 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:52:56.768764 12362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.768448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:46205 (local address 127.0.0.1:44930)
0810 18:52:56.768570 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:56.768657 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.188:46205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 18:52:56.768983 12301 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.168.188:46205: connect: Connection refused (error 111)
I20260810 18:52:56.769942 12362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.769702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:46205 (local address 127.0.0.1:44934)
0810 18:52:56.769817 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:56.769880 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.188:46205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260810 18:52:56.770167 12276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.759994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.189:33023 (local address 127.0.0.1:46846)
0810 18:52:56.760123 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:56.760138 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:56.760235 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:56.761397 (+ 1162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:56.761400 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:56.761568 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:52:56.762220 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:56.762234 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:56.763104 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:56.763109 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:56.764071 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:56.764081 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:56.764542 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:56.764545 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:56.764654 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:56.765081 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:52:56.765097 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:52:56.766485 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:52:56.768531 (+ 2046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:56.768534 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:56.768536 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:56.768871 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:56.769176 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:56.769178 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:56.769180 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:56.769251 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:56.769673 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:52:56.769676 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:52:56.769778 (+ 102us) client_negotiation.cc:769] Sending connection context
0810 18:52:56.769886 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 18:52:56.769984 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
W20260810 18:52:56.770174 12360 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.168.188:46205: Network error: Client connection negotiation failed: client connection to 127.18.168.188:46205: connect: Connection refused (error 111)
I20260810 18:52:56.770306 12367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.760754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46846 (local address 127.18.168.189:33023)
0810 18:52:56.761103 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:56.761112 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:56.761128 (+ 16us) server_negotiation.cc:407] Connection header received
0810 18:52:56.761169 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:56.761172 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:56.761210 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:56.761358 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:56.762348 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:56.762992 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:56.764183 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:56.764433 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:56.764537 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:56.766626 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:56.766629 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:56.766632 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:56.766656 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:56.768412 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:56.768990 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:56.768994 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:56.768997 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:56.769086 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:56.769360 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:56.769363 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:56.769365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:56.769570 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:56.769679 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:56.769964 (+ 285us) server_negotiation.cc:299] Negotiation successful
0810 18:52:56.770099 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":111,"threads_started":1}
I20260810 18:52:56.771005 12273 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46205 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.188.keytab.
Entry for principal kudu/127.18.168.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.188.keytab.
I20260810 18:52:56.771747 12276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.771501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:46205 (local address 127.0.0.1:44942)
0810 18:52:56.771619 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:56.771681 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.188:46205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 18:52:56.772392 12276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.772162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:46205 (local address 127.0.0.1:44950)
0810 18:52:56.772296 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:56.772339 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.188:46205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260810 18:52:56.772552 12273 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.168.188:46205: Network error: Client connection negotiation failed: client connection to 127.18.168.188:46205: connect: Connection refused (error 111)
I20260810 18:52:56.772575 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.188:46205
--webserver_interface=127.18.168.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.188.keytab
--principal=kudu/127.18.168.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:40257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:43139,127.18.168.189:33023,127.18.168.188:46205
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:52:56.803735 12273 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46205 } attempt: 1
I20260810 18:52:56.804622 12276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.804265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:46205 (local address 127.0.0.1:44954)
0810 18:52:56.804434 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:56.804503 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.188:46205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260810 18:52:56.805384 12276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.805153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:46205 (local address 127.0.0.1:44958)
0810 18:52:56.805270 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:56.805323 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.188:46205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260810 18:52:56.805536 12273 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.168.188:46205: Network error: Client connection negotiation failed: client connection to 127.18.168.188:46205: connect: Connection refused (error 111)
I20260810 18:52:56.819432 12360 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46205 } attempt: 1
I20260810 18:52:56.820293 12362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.820021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:46205 (local address 127.0.0.1:44970)
0810 18:52:56.820161 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:56.820219 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.188:46205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 18:52:56.821017 12362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.820817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:46205 (local address 127.0.0.1:44986)
0810 18:52:56.820919 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:56.820967 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.188:46205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260810 18:52:56.821148 12360 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.168.188:46205: Network error: Client connection negotiation failed: client connection to 127.18.168.188:46205: connect: Connection refused (error 111)
I20260810 18:52:56.880743 12273 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46205 } attempt: 2
I20260810 18:52:56.881668 12276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.881343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:46205 (local address 127.0.0.1:44988)
0810 18:52:56.881504 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:56.881571 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.188:46205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 18:52:56.882409 12276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.882190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:46205 (local address 127.0.0.1:45000)
0810 18:52:56.882295 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:56.882345 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.188:46205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260810 18:52:56.882550 12273 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.168.188:46205: Network error: Client connection negotiation failed: client connection to 127.18.168.188:46205: connect: Connection refused (error 111)
W20260810 18:52:56.886523 12369 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 18:52:56.886811 12369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:56.886870 12369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:56.886942 12369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260810 18:52:56.889310 12360 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46205 } attempt: 2
I20260810 18:52:56.890061 12362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.889783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:46205 (local address 127.0.0.1:45004)
0810 18:52:56.889939 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:56.889998 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.188:46205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260810 18:52:56.890769 12362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.890553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:46205 (local address 127.0.0.1:45012)
0810 18:52:56.890665 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:56.890710 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.188:46205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260810 18:52:56.890892 12360 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.168.188:46205: Network error: Client connection negotiation failed: client connection to 127.18.168.188:46205: connect: Connection refused (error 111)
W20260810 18:52:56.891368 12369 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:52:56.891441 12369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:56.891503 12369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:56.891541 12369 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:52:56.891579 12369 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:52:56.896857 12369 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:40257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:43139,127.18.168.189:33023,127.18.168.188:46205
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.188.keytab
--principal=kudu/127.18.168.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.188:46205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.168.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:56.897900 12369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:56.898895 12369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:56.907208 12376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:56.907215 12379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:56.907218 12377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:56.907554 12369 server_base.cc:1034] running on GCE node
I20260810 18:52:56.908049 12369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:56.909211 12369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:56.910409 12369 hybrid_clock.cc:648] HybridClock initialized: now 1786387976910392 us; error 30 us; skew 500 ppm
Aug 10 18:52:56 dist-test-slave-cbsf krb5kdc[12183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387976, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:56.913079 12369 init.cc:377] Logged in from keytab as kudu/127.18.168.188@KRBTEST.COM (short username kudu)
I20260810 18:52:56.913893 12369 webserver.cc:460] Webserver started at http://127.18.168.188:36265/ using document root <none> and password file <none>
I20260810 18:52:56.914400 12369 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:56.914460 12369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:56.914630 12369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:56.916545 12369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-2/data/instance:
uuid: "04c60618ac8f4866bb9125f6c6c1c98d"
format_stamp: "Formatted at 2026-08-10 18:52:56 on dist-test-slave-cbsf"
server_key: "6e8785b0df53ca9e69c62f1b568891db"
server_key_iv: "6b7bbe468ca4501737ee2e8c8ff450d5"
server_key_version: "encryptionkey@0"
I20260810 18:52:56.917130 12369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-2/wal/instance:
uuid: "04c60618ac8f4866bb9125f6c6c1c98d"
format_stamp: "Formatted at 2026-08-10 18:52:56 on dist-test-slave-cbsf"
server_key: "6e8785b0df53ca9e69c62f1b568891db"
server_key_iv: "6b7bbe468ca4501737ee2e8c8ff450d5"
server_key_version: "encryptionkey@0"
I20260810 18:52:56.920598 12369 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:52:56.922884 12386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:56.923980 12369 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:52:56.924103 12369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-2/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-2/wal
uuid: "04c60618ac8f4866bb9125f6c6c1c98d"
format_stamp: "Formatted at 2026-08-10 18:52:56 on dist-test-slave-cbsf"
server_key: "6e8785b0df53ca9e69c62f1b568891db"
server_key_iv: "6b7bbe468ca4501737ee2e8c8ff450d5"
server_key_version: "encryptionkey@0"
I20260810 18:52:56.924204 12369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:56.941464 12369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:56.944053 12369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:56.944257 12369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:56.951232 12369 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.188:46205
I20260810 18:52:56.951519 12442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.188:46205 every 8 connection(s)
I20260810 18:52:56.952184 12369 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-2/data/info.pb
I20260810 18:52:56.954911 12443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:56.958963 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 12369
I20260810 18:52:56.959100 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387687857264-19106-0/minicluster-data/master-2/wal/instance
I20260810 18:52:56.958863 12443 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43139 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:52:56.959451 19106 external_mini_cluster.cc:1428] Setting key 44adaf9af579e0b443ec05317ca2bbf1
Aug 10 18:52:56 dist-test-slave-cbsf krb5kdc[12183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387976, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.188@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
Aug 10 18:52:56 dist-test-slave-cbsf krb5kdc[12183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387976, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.168.188@KRBTEST.COM
I20260810 18:52:56.973210 12277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.960936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42446 (local address 127.18.168.190:43139)
0810 18:52:56.961100 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:56.961106 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:56.961508 (+ 402us) server_negotiation.cc:407] Connection header received
0810 18:52:56.962132 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:56.962136 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:56.962182 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:56.962267 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:56.963417 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:56.964212 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:56.965627 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:56.965957 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:56.966064 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:56.969062 (+ 2998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:56.969069 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:56.969075 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:56.969110 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:56.970879 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:56.971500 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:56.971503 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:56.971505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:56.971591 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:56.971969 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:56.971972 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:56.971974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:56.972198 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:56.972264 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:56.972933 (+ 669us) server_negotiation.cc:299] Negotiation successful
0810 18:52:56.973055 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 18:52:56.974619 12447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.960886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:43139 (local address 127.0.0.1:42446)
0810 18:52:56.961350 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:56.961395 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:56.961931 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:56.962335 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:56.962339 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:56.962863 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:52:56.963256 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:56.963280 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:56.964326 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:56.964329 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:56.965502 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:56.965513 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:56.966099 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:56.966114 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:56.966250 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:56.967023 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:52:56.967044 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:52:56.968827 (+ 1783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:52:56.971008 (+ 2181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:56.971021 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:56.971032 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:56.971376 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:56.971695 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:56.971698 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:56.971699 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:56.971852 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:56.972343 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:52:56.972349 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:52:56.972632 (+ 283us) client_negotiation.cc:769] Sending connection context
0810 18:52:56.973359 (+ 727us) client_negotiation.cc:240] Negotiation successful
0810 18:52:56.973693 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"mutex_wait_us":42,"spinlock_wait_cycles":18688,"thread_start_us":195,"threads_started":1}
I20260810 18:52:56.974623 12446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.960799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45024 (local address 127.18.168.188:46205)
0810 18:52:56.961189 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:56.961205 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:56.961237 (+ 32us) server_negotiation.cc:407] Connection header received
0810 18:52:56.961916 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:56.961932 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:56.962143 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:56.962309 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:56.963250 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:56.964455 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:56.965627 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:56.966191 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:56.966292 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:56.969493 (+ 3201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:56.969503 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:56.969519 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:56.969548 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:56.971844 (+ 2296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:56.972360 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:56.972366 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:56.972368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:56.972448 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:56.972758 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:56.972761 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:56.972762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:56.973011 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:56.973869 (+ 858us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:56.973966 (+ 97us) server_negotiation.cc:299] Negotiation successful
0810 18:52:56.974094 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":253,"thread_start_us":118,"threads_started":1}
I20260810 18:52:56.979506 12443 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33023 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:52:56.981729 12273 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46205 } attempt: 3
I20260810 18:52:56.982082 12360 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46205 } attempt: 3
Aug 10 18:52:56 dist-test-slave-cbsf krb5kdc[12183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387976, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.188@KRBTEST.COM for kudu/127.18.168.189@KRBTEST.COM
Aug 10 18:52:56 dist-test-slave-cbsf krb5kdc[12183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387976, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
Aug 10 18:52:56 dist-test-slave-cbsf krb5kdc[12183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387976, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for kudu/127.18.168.188@KRBTEST.COM
I20260810 18:52:56.989581 12367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.980112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46858 (local address 127.18.168.189:33023)
0810 18:52:56.980277 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:56.980282 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:56.980296 (+ 14us) server_negotiation.cc:407] Connection header received
0810 18:52:56.980355 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:56.980358 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:56.980403 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:56.980480 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:56.980948 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:56.981478 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:56.982457 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:56.982811 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:56.982895 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:56.985513 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:56.985517 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:56.985521 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:56.985550 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:56.987400 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:56.987968 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:56.987972 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:56.987974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:56.988064 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:56.988532 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:56.988535 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:56.988537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:56.988779 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:56.988892 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:56.989266 (+ 374us) server_negotiation.cc:299] Negotiation successful
0810 18:52:56.989401 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 18:52:56.990093 12447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.979991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.189:33023 (local address 127.0.0.1:46858)
0810 18:52:56.980149 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:56.980161 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:56.980221 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:56.980508 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:56.980511 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:56.980665 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:52:56.980819 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:56.980825 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:56.981586 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:56.981590 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:56.982318 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:56.982328 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:56.983039 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:56.983044 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:56.983167 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:56.983661 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:52:56.983674 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:52:56.985311 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:52:56.987547 (+ 2236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:56.987553 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:56.987558 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:56.987838 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:56.988189 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:56.988194 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:56.988198 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:56.988308 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:56.988956 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:52:56.988960 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:52:56.989068 (+ 108us) client_negotiation.cc:769] Sending connection context
0810 18:52:56.989782 (+ 714us) client_negotiation.cc:240] Negotiation successful
0810 18:52:56.989913 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94}
I20260810 18:52:56.990790 12443 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46205 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:52:56 dist-test-slave-cbsf krb5kdc[12183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387976, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.189@KRBTEST.COM for kudu/127.18.168.188@KRBTEST.COM
I20260810 18:52:56.992820 12277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.978844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42456 (local address 127.18.168.190:43139)
0810 18:52:56.978966 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:56.978972 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:56.978986 (+ 14us) server_negotiation.cc:407] Connection header received
0810 18:52:56.979024 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:56.979027 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:56.979066 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:56.979141 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:56.979983 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:56.980900 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:56.981758 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:56.982246 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:56.982931 (+ 685us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:56.986399 (+ 3468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:56.986406 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:56.986410 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:56.986455 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:56.988943 (+ 2488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:56.989475 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:56.989478 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:56.989480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:56.989552 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:56.990239 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:56.990243 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:56.990247 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:56.991238 (+ 991us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:56.991634 (+ 396us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:56.992300 (+ 666us) server_negotiation.cc:299] Negotiation successful
0810 18:52:56.992419 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 18:52:56.993868 12276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.982193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:46205 (local address 127.0.0.1:45032)
0810 18:52:56.982341 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:56.982356 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:56.982428 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:56.983160 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:56.983163 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:56.983356 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:52:56.983443 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:56.983448 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:56.984584 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:56.984590 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:56.985507 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:56.985517 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:56.986357 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:56.986363 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:56.986461 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:56.986873 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:52:56.986890 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:52:56.988653 (+ 1763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:52:56.991847 (+ 3194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:56.991850 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:56.991852 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:56.992134 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:56.992459 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:56.992474 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:56.992477 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:56.992606 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:56.993377 (+ 771us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:52:56.993383 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:52:56.993476 (+ 93us) client_negotiation.cc:769] Sending connection context
0810 18:52:56.993579 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 18:52:56.993698 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260810 18:52:56.994537 12448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.982570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45032 (local address 127.18.168.188:46205)
0810 18:52:56.982921 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:56.982930 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:56.982943 (+ 13us) server_negotiation.cc:407] Connection header received
0810 18:52:56.982979 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:56.982985 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:56.983028 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:56.983569 (+ 541us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:56.983671 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:56.984444 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:56.985708 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:56.986204 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:56.986682 (+ 478us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:56.989081 (+ 2399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:56.989084 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:56.989087 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:56.989106 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:56.991706 (+ 2600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:56.992263 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:56.992266 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:56.992268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:56.992349 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:56.992731 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:56.992734 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:56.992735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:56.993267 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:56.994160 (+ 893us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:56.994237 (+ 77us) server_negotiation.cc:299] Negotiation successful
0810 18:52:56.994347 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":133,"threads_started":1}
Aug 10 18:52:56 dist-test-slave-cbsf krb5kdc[12183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387976, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.188@KRBTEST.COM for kudu/127.18.168.188@KRBTEST.COM
I20260810 18:52:56.998548 12273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 980384115e344d769dce6dd222d03a8c: Bootstrap starting.
I20260810 18:52:57.001119 12447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.991290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:46205 (local address 127.0.0.1:45042)
0810 18:52:56.991716 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:56.991731 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:56.991828 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:56.992981 (+ 1153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:56.992984 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:56.993147 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:52:56.993230 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:56.993236 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:56.994227 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:56.994231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:56.995024 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:56.995035 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:56.995731 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:56.995734 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:56.995800 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:56.996283 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:52:56.996297 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:52:56.997674 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:52:56.999638 (+ 1964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:56.999641 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:56.999643 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:56.999857 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:57.000127 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:57.000132 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:57.000135 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:57.000208 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:57.000668 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:52:57.000671 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:52:57.000764 (+ 93us) client_negotiation.cc:769] Sending connection context
0810 18:52:57.000856 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 18:52:57.000985 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355}
I20260810 18:52:57.001564 12449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.991485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45042 (local address 127.18.168.188:46205)
0810 18:52:56.992636 (+ 1151us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:56.992643 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:56.992652 (+ 9us) server_negotiation.cc:407] Connection header received
0810 18:52:56.992727 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:56.992729 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:56.992769 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:56.992853 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:56.993348 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:56.994123 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:56.995168 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:56.995539 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:56.995636 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:56.997805 (+ 2169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:56.997808 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:56.997811 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:56.997832 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:56.999532 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:56.999958 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:56.999963 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:56.999964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:57.000042 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:57.000291 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:57.000294 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:57.000295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:57.000526 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:57.000637 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:57.001301 (+ 664us) server_negotiation.cc:299] Negotiation successful
0810 18:52:57.001402 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1098,"thread_start_us":116,"threads_started":1}
I20260810 18:52:57.001729 12273 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 980384115e344d769dce6dd222d03a8c: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:57.002631 12273 log.cc:824] T 00000000000000000000000000000000 P 980384115e344d769dce6dd222d03a8c: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:57.005128 12273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 980384115e344d769dce6dd222d03a8c: No bootstrap required, opened a new log
I20260810 18:52:57.005707 12443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04c60618ac8f4866bb9125f6c6c1c98d: Bootstrap starting.
I20260810 18:52:57.006747 12362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.982452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:46205 (local address 127.0.0.1:45036)
0810 18:52:56.982640 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:56.982658 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:56.982744 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:56.983225 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:56.983228 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:56.983989 (+ 761us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:52:56.984108 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:56.984118 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:56.986324 (+ 2206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:56.986328 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:56.988327 (+ 1999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:56.988341 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:56.989008 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:56.989012 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:56.989099 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:56.989776 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:52:56.989791 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:52:56.998362 (+ 8571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:52:57.005074 (+ 6712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:57.005077 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:57.005079 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:57.005336 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:57.005722 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:57.005725 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:57.005727 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:57.005822 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:57.006221 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:52:57.006224 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:52:57.006311 (+ 87us) client_negotiation.cc:769] Sending connection context
0810 18:52:57.006394 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 18:52:57.006592 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104}
I20260810 18:52:57.007308 12446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.982487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45036 (local address 127.18.168.188:46205)
0810 18:52:56.982849 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:56.982856 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:56.982876 (+ 20us) server_negotiation.cc:407] Connection header received
0810 18:52:56.982915 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:56.982918 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:56.983050 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:56.984248 (+ 1198us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:56.984366 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:56.986170 (+ 1804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:56.988487 (+ 2317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:56.988866 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:56.989683 (+ 817us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:57.003133 (+ 13450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:57.003136 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:57.003150 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:57.003172 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:57.004943 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:57.005522 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:57.005526 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:57.005528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:57.005614 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:57.005938 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:57.005941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:57.005942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:57.006112 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:57.006985 (+ 873us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:57.007076 (+ 91us) server_negotiation.cc:299] Negotiation successful
0810 18:52:57.007179 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275}
Aug 10 18:52:57 dist-test-slave-cbsf krb5kdc[12183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387976, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.168.189@KRBTEST.COM
I20260810 18:52:57.008698 12443 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04c60618ac8f4866bb9125f6c6c1c98d: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:57.009738 12443 log.cc:824] T 00000000000000000000000000000000 P 04c60618ac8f4866bb9125f6c6c1c98d: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:57.009820 12273 raft_consensus.cc:348] T 00000000000000000000000000000000 P 980384115e344d769dce6dd222d03a8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "980384115e344d769dce6dd222d03a8c" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43139 } } peers { permanent_uuid: "90a8c9bc1fea49cfb4ca1ed1e9eff9c6" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33023 } } peers { permanent_uuid: "04c60618ac8f4866bb9125f6c6c1c98d" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46205 } }
I20260810 18:52:57.010083 12273 raft_consensus.cc:729] T 00000000000000000000000000000000 P 980384115e344d769dce6dd222d03a8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 980384115e344d769dce6dd222d03a8c, State: Initialized, Role: FOLLOWER
I20260810 18:52:57.010125 12360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90a8c9bc1fea49cfb4ca1ed1e9eff9c6: Bootstrap starting.
I20260810 18:52:57.010396 12273 consensus_queue.cc:260] T 00000000000000000000000000000000 P 980384115e344d769dce6dd222d03a8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "980384115e344d769dce6dd222d03a8c" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43139 } } peers { permanent_uuid: "90a8c9bc1fea49cfb4ca1ed1e9eff9c6" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33023 } } peers { permanent_uuid: "04c60618ac8f4866bb9125f6c6c1c98d" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46205 } }
I20260810 18:52:57.010927 12454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 980384115e344d769dce6dd222d03a8c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "980384115e344d769dce6dd222d03a8c" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43139 } } peers { permanent_uuid: "90a8c9bc1fea49cfb4ca1ed1e9eff9c6" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33023 } } peers { permanent_uuid: "04c60618ac8f4866bb9125f6c6c1c98d" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46205 } } }
I20260810 18:52:57.011170 12454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 980384115e344d769dce6dd222d03a8c [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:52:57.011750 12273 sys_catalog.cc:564] T 00000000000000000000000000000000 P 980384115e344d769dce6dd222d03a8c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:52:57.012548 12360 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90a8c9bc1fea49cfb4ca1ed1e9eff9c6: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:57.013456 12367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.997592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46872 (local address 127.18.168.189:33023)
0810 18:52:56.997856 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:56.997867 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:56.997885 (+ 18us) server_negotiation.cc:407] Connection header received
0810 18:52:56.997984 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:56.997987 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:56.998071 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:56.999532 (+ 1461us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:56.999697 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:57.000853 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:57.002790 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:57.005068 (+ 2278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:57.005607 (+ 539us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:57.009328 (+ 3721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:57.009332 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:57.009335 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:57.009362 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:57.011452 (+ 2090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:57.011959 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:57.011966 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:57.011971 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:57.012074 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:57.012389 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:57.012392 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:57.012394 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:57.012668 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:57.012937 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:57.013158 (+ 221us) server_negotiation.cc:299] Negotiation successful
0810 18:52:57.013282 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 18:52:57.013447 12360 log.cc:824] T 00000000000000000000000000000000 P 90a8c9bc1fea49cfb4ca1ed1e9eff9c6: Log is configured to *not* fsync() on all Append() calls
W20260810 18:52:57.013859 12220 tablet.cc:2369] T 00000000000000000000000000000000 P 980384115e344d769dce6dd222d03a8c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:52:57.017103 19106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260810 18:52:57.017189 12466 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 980384115e344d769dce6dd222d03a8c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:52:57.017262 12466 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:52:57.018563 12245 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "980384115e344d769dce6dd222d03a8c"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42456
I20260810 18:52:57.018769 12245 raft_consensus.cc:482] T 00000000000000000000000000000000 P 980384115e344d769dce6dd222d03a8c [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 18:52:57.018872 12245 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 980384115e344d769dce6dd222d03a8c [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:57.019110 12443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04c60618ac8f4866bb9125f6c6c1c98d: No bootstrap required, opened a new log
I20260810 18:52:57.020128 12245 raft_consensus.cc:504] T 00000000000000000000000000000000 P 980384115e344d769dce6dd222d03a8c [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "980384115e344d769dce6dd222d03a8c" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43139 } } peers { permanent_uuid: "90a8c9bc1fea49cfb4ca1ed1e9eff9c6" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33023 } } peers { permanent_uuid: "04c60618ac8f4866bb9125f6c6c1c98d" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46205 } }
I20260810 18:52:57.021050 12245 leader_election.cc:288] T 00000000000000000000000000000000 P 980384115e344d769dce6dd222d03a8c [CANDIDATE]: Term 1 election: Requested vote from peers 90a8c9bc1fea49cfb4ca1ed1e9eff9c6 (127.18.168.189:33023), 04c60618ac8f4866bb9125f6c6c1c98d (127.18.168.188:46205)
I20260810 18:52:57.017704 12197 master_runner.cc:186] Error getting master registration for 127.18.168.189:33023: OK, instance_id { permanent_uuid: "90a8c9bc1fea49cfb4ca1ed1e9eff9c6" instance_seqno: 1786387976729799 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 18:52:57.021385 12197 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 18:52:57.023815 12443 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04c60618ac8f4866bb9125f6c6c1c98d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "980384115e344d769dce6dd222d03a8c" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43139 } } peers { permanent_uuid: "90a8c9bc1fea49cfb4ca1ed1e9eff9c6" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33023 } } peers { permanent_uuid: "04c60618ac8f4866bb9125f6c6c1c98d" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46205 } }
I20260810 18:52:57.024072 12443 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04c60618ac8f4866bb9125f6c6c1c98d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04c60618ac8f4866bb9125f6c6c1c98d, State: Initialized, Role: FOLLOWER
I20260810 18:52:57.024398 12443 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04c60618ac8f4866bb9125f6c6c1c98d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "980384115e344d769dce6dd222d03a8c" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43139 } } peers { permanent_uuid: "90a8c9bc1fea49cfb4ca1ed1e9eff9c6" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33023 } } peers { permanent_uuid: "04c60618ac8f4866bb9125f6c6c1c98d" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46205 } }
I20260810 18:52:57.025005 12476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04c60618ac8f4866bb9125f6c6c1c98d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "980384115e344d769dce6dd222d03a8c" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43139 } } peers { permanent_uuid: "90a8c9bc1fea49cfb4ca1ed1e9eff9c6" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33023 } } peers { permanent_uuid: "04c60618ac8f4866bb9125f6c6c1c98d" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46205 } } }
I20260810 18:52:57.025307 12476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04c60618ac8f4866bb9125f6c6c1c98d [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:52:57.025774 12443 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04c60618ac8f4866bb9125f6c6c1c98d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:52:57.025802 12415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "980384115e344d769dce6dd222d03a8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "04c60618ac8f4866bb9125f6c6c1c98d"
I20260810 18:52:57.025980 12415 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04c60618ac8f4866bb9125f6c6c1c98d [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:57.027446 12415 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 04c60618ac8f4866bb9125f6c6c1c98d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 980384115e344d769dce6dd222d03a8c in term 1.
I20260810 18:52:57.027913 12216 leader_election.cc:302] T 00000000000000000000000000000000 P 980384115e344d769dce6dd222d03a8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04c60618ac8f4866bb9125f6c6c1c98d, 980384115e344d769dce6dd222d03a8c; no voters:
I20260810 18:52:57.028268 12454 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 980384115e344d769dce6dd222d03a8c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:52:57.028532 12454 raft_consensus.cc:686] T 00000000000000000000000000000000 P 980384115e344d769dce6dd222d03a8c [term 1 LEADER]: Becoming Leader. State: Replica: 980384115e344d769dce6dd222d03a8c, State: Running, Role: LEADER
I20260810 18:52:57.028880 12454 consensus_queue.cc:237] T 00000000000000000000000000000000 P 980384115e344d769dce6dd222d03a8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "980384115e344d769dce6dd222d03a8c" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43139 } } peers { permanent_uuid: "90a8c9bc1fea49cfb4ca1ed1e9eff9c6" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33023 } } peers { permanent_uuid: "04c60618ac8f4866bb9125f6c6c1c98d" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46205 } }
I20260810 18:52:57.030025 12360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90a8c9bc1fea49cfb4ca1ed1e9eff9c6: No bootstrap required, opened a new log
I20260810 18:52:57.030580 12454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 980384115e344d769dce6dd222d03a8c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 980384115e344d769dce6dd222d03a8c. Latest consensus state: current_term: 1 leader_uuid: "980384115e344d769dce6dd222d03a8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "980384115e344d769dce6dd222d03a8c" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43139 } } peers { permanent_uuid: "90a8c9bc1fea49cfb4ca1ed1e9eff9c6" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33023 } } peers { permanent_uuid: "04c60618ac8f4866bb9125f6c6c1c98d" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46205 } } }
I20260810 18:52:57.030694 12454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 980384115e344d769dce6dd222d03a8c [sys.catalog]: This master's current role is: LEADER
W20260810 18:52:57.031322 12487 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 04c60618ac8f4866bb9125f6c6c1c98d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:52:57.031412 12487 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:52:57.033110 12491 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:52:57.034184 12491 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:52:57.035404 12369 master_runner.cc:186] Error getting master registration for 127.18.168.189:33023: OK, instance_id { permanent_uuid: "90a8c9bc1fea49cfb4ca1ed1e9eff9c6" instance_seqno: 1786387976729799 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 18:52:57.038807 12369 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 18:52:57.038813 12360 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90a8c9bc1fea49cfb4ca1ed1e9eff9c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "980384115e344d769dce6dd222d03a8c" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43139 } } peers { permanent_uuid: "90a8c9bc1fea49cfb4ca1ed1e9eff9c6" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33023 } } peers { permanent_uuid: "04c60618ac8f4866bb9125f6c6c1c98d" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46205 } }
I20260810 18:52:57.039055 12360 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90a8c9bc1fea49cfb4ca1ed1e9eff9c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90a8c9bc1fea49cfb4ca1ed1e9eff9c6, State: Initialized, Role: FOLLOWER
I20260810 18:52:57.039397 12360 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90a8c9bc1fea49cfb4ca1ed1e9eff9c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "980384115e344d769dce6dd222d03a8c" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43139 } } peers { permanent_uuid: "90a8c9bc1fea49cfb4ca1ed1e9eff9c6" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33023 } } peers { permanent_uuid: "04c60618ac8f4866bb9125f6c6c1c98d" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46205 } }
I20260810 18:52:57.040812 12367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:57.023729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46886 (local address 127.18.168.189:33023)
0810 18:52:57.023839 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:57.023844 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:57.024892 (+ 1048us) server_negotiation.cc:407] Connection header received
0810 18:52:57.025099 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:57.025103 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:57.025152 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:57.025206 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:57.026251 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:57.027147 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:57.029236 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:57.029567 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:57.029650 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:57.033188 (+ 3538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:57.033192 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:57.033195 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:57.033219 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:57.034980 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:57.039334 (+ 4354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:57.039338 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:57.039340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:57.039434 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:57.039789 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:57.039793 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:57.039795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:57.040027 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:57.040145 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:57.040512 (+ 367us) server_negotiation.cc:299] Negotiation successful
0810 18:52:57.040631 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 18:52:57.040915 12415 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 04c60618ac8f4866bb9125f6c6c1c98d [term 1 FOLLOWER]: Refusing update from remote peer 980384115e344d769dce6dd222d03a8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:52:57.042033 12488 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 980384115e344d769dce6dd222d03a8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "04c60618ac8f4866bb9125f6c6c1c98d" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:52:57.042819 12360 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90a8c9bc1fea49cfb4ca1ed1e9eff9c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:52:57.043192 12330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "980384115e344d769dce6dd222d03a8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "90a8c9bc1fea49cfb4ca1ed1e9eff9c6"
I20260810 18:52:57.043403 12330 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90a8c9bc1fea49cfb4ca1ed1e9eff9c6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:57.044795 12330 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 90a8c9bc1fea49cfb4ca1ed1e9eff9c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 980384115e344d769dce6dd222d03a8c in term 1.
I20260810 18:52:57.045233 12328 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 90a8c9bc1fea49cfb4ca1ed1e9eff9c6 [term 1 FOLLOWER]: Refusing update from remote peer 980384115e344d769dce6dd222d03a8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:52:57.048017 12277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:57.024269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42458 (local address 127.18.168.190:43139)
0810 18:52:57.024394 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:57.024399 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:57.030505 (+ 6106us) server_negotiation.cc:407] Connection header received
0810 18:52:57.030733 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:57.030737 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:57.030794 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:57.030850 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:57.031911 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:57.032618 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:57.035109 (+ 2491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:57.035415 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:57.035535 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:57.043781 (+ 8246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:57.043788 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:57.043795 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:57.043832 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:57.045941 (+ 2109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:57.046481 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:57.046485 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:57.046487 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:57.046580 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:57.046929 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:57.046932 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:57.046934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:57.047177 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:57.047301 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:57.047694 (+ 393us) server_negotiation.cc:299] Negotiation successful
0810 18:52:57.047833 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 18:52:57.049906 12488 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 980384115e344d769dce6dd222d03a8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "90a8c9bc1fea49cfb4ca1ed1e9eff9c6" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:52:57.052886 12493 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90a8c9bc1fea49cfb4ca1ed1e9eff9c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "980384115e344d769dce6dd222d03a8c" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43139 } } peers { permanent_uuid: "90a8c9bc1fea49cfb4ca1ed1e9eff9c6" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33023 } } peers { permanent_uuid: "04c60618ac8f4866bb9125f6c6c1c98d" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46205 } } }
I20260810 18:52:57.053018 12493 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90a8c9bc1fea49cfb4ca1ed1e9eff9c6 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:52:57.053700 12476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04c60618ac8f4866bb9125f6c6c1c98d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 980384115e344d769dce6dd222d03a8c. Latest consensus state: current_term: 1 leader_uuid: "980384115e344d769dce6dd222d03a8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "980384115e344d769dce6dd222d03a8c" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43139 } } peers { permanent_uuid: "90a8c9bc1fea49cfb4ca1ed1e9eff9c6" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33023 } } peers { permanent_uuid: "04c60618ac8f4866bb9125f6c6c1c98d" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46205 } } }
I20260810 18:52:57.053812 12476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04c60618ac8f4866bb9125f6c6c1c98d [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:52:57.059610 12493 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90a8c9bc1fea49cfb4ca1ed1e9eff9c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 980384115e344d769dce6dd222d03a8c. Latest consensus state: current_term: 1 leader_uuid: "980384115e344d769dce6dd222d03a8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "980384115e344d769dce6dd222d03a8c" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43139 } } peers { permanent_uuid: "90a8c9bc1fea49cfb4ca1ed1e9eff9c6" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33023 } } peers { permanent_uuid: "04c60618ac8f4866bb9125f6c6c1c98d" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46205 } } }
I20260810 18:52:57.059722 12493 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90a8c9bc1fea49cfb4ca1ed1e9eff9c6 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:52:57.060595 12476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04c60618ac8f4866bb9125f6c6c1c98d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "980384115e344d769dce6dd222d03a8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "980384115e344d769dce6dd222d03a8c" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43139 } } peers { permanent_uuid: "90a8c9bc1fea49cfb4ca1ed1e9eff9c6" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33023 } } peers { permanent_uuid: "04c60618ac8f4866bb9125f6c6c1c98d" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46205 } } }
I20260810 18:52:57.060727 12476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04c60618ac8f4866bb9125f6c6c1c98d [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:52:57.061801 12454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 980384115e344d769dce6dd222d03a8c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "980384115e344d769dce6dd222d03a8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "980384115e344d769dce6dd222d03a8c" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43139 } } peers { permanent_uuid: "90a8c9bc1fea49cfb4ca1ed1e9eff9c6" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33023 } } peers { permanent_uuid: "04c60618ac8f4866bb9125f6c6c1c98d" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46205 } } }
I20260810 18:52:57.061904 12454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 980384115e344d769dce6dd222d03a8c [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:57.062016 12492 mvcc.cc:205] Tried to move back new op lower bound from 7317045153949429760 to 7317045153913663488. Current Snapshot: MvccSnapshot[applied={T|T < 7317045153949429760}]
I20260810 18:52:57.066627 12488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 980384115e344d769dce6dd222d03a8c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "980384115e344d769dce6dd222d03a8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "980384115e344d769dce6dd222d03a8c" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43139 } } peers { permanent_uuid: "90a8c9bc1fea49cfb4ca1ed1e9eff9c6" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33023 } } peers { permanent_uuid: "04c60618ac8f4866bb9125f6c6c1c98d" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46205 } } }
I20260810 18:52:57.066764 12488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 980384115e344d769dce6dd222d03a8c [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:57.067757 12493 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90a8c9bc1fea49cfb4ca1ed1e9eff9c6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "980384115e344d769dce6dd222d03a8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "980384115e344d769dce6dd222d03a8c" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 43139 } } peers { permanent_uuid: "90a8c9bc1fea49cfb4ca1ed1e9eff9c6" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 33023 } } peers { permanent_uuid: "04c60618ac8f4866bb9125f6c6c1c98d" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 46205 } } }
I20260810 18:52:57.067885 12493 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90a8c9bc1fea49cfb4ca1ed1e9eff9c6 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:52:57.069911 12446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:57.053312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45054 (local address 127.18.168.188:46205)
0810 18:52:57.053411 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:57.053416 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:57.053429 (+ 13us) server_negotiation.cc:407] Connection header received
0810 18:52:57.053587 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:57.053591 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:57.053641 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:57.053720 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:57.054803 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:57.055702 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:57.057016 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:57.057461 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:57.057548 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:57.059458 (+ 1910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:57.059463 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:57.059466 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:57.059494 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:57.067160 (+ 7666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:57.067805 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:57.067809 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:57.067811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:57.068023 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:57.068379 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:57.068383 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:57.068384 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:57.068647 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:57.068837 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:57.069593 (+ 756us) server_negotiation.cc:299] Negotiation successful
0810 18:52:57.069726 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260810 18:52:57.071599 12504 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 90a8c9bc1fea49cfb4ca1ed1e9eff9c6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:52:57.071683 12504 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:52:57.073662 12508 mvcc.cc:205] Tried to move back new op lower bound from 7317045153949429760 to 7317045153913663488. Current Snapshot: MvccSnapshot[applied={T|T < 7317045153949429760}]
I20260810 18:52:57.077536 12491 catalog_manager.cc:1347] Generated new cluster ID: 13d5570948a84cb1842be1f01b48e870
I20260810 18:52:57.077610 12491 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:52:57.077869 12505 mvcc.cc:205] Tried to move back new op lower bound from 7317045153949429760 to 7317045153913663488. Current Snapshot: MvccSnapshot[applied={T|T < 7317045153949429760}]
I20260810 18:52:57.093930 12491 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:52:57.095232 12491 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:52:57.109692 12491 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 980384115e344d769dce6dd222d03a8c: Generated new TSK 0
I20260810 18:52:57.110502 12491 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 18:52:57.150007 12277 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:52:57.146379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42460 (local address 127.18.168.190:43139)
0810 18:52:57.146533 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:57.146538 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:57.148829 (+ 2291us) server_negotiation.cc:407] Connection header received
0810 18:52:57.149043 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:57.149051 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:57.149106 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:57.149163 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:57.149853 (+ 690us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42460: BlockingRecv error: recv got EOF from 127.0.0.1:42460 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260810 18:52:57.151743 12446 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:52:57.147310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45062 (local address 127.18.168.188:46205)
0810 18:52:57.147432 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:57.147437 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:57.150769 (+ 3332us) server_negotiation.cc:407] Connection header received
0810 18:52:57.150947 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:57.150950 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:57.151022 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:57.151077 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:57.151621 (+ 544us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45062: BlockingRecv error: recv got EOF from 127.0.0.1:45062 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260810 18:52:57.153519 12367 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:52:57.148094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46896 (local address 127.18.168.189:33023)
0810 18:52:57.148214 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:57.148219 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:57.152382 (+ 4163us) server_negotiation.cc:407] Connection header received
0810 18:52:57.152666 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:57.152670 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:57.152796 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:57.152855 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:57.153378 (+ 523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46896: BlockingRecv error: recv got EOF from 127.0.0.1:46896 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260810 18:52:57.158393 12367 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:52:57.156757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46910 (local address 127.18.168.189:33023)
0810 18:52:57.156863 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:57.156868 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:57.157349 (+ 481us) server_negotiation.cc:407] Connection header received
0810 18:52:57.157617 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:57.157620 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:57.157695 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:57.157749 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:57.158279 (+ 530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46910: BlockingRecv error: recv got EOF from 127.0.0.1:46910 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260810 18:52:57.163663 12367 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:52:57.162211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46918 (local address 127.18.168.189:33023)
0810 18:52:57.162308 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:57.162312 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:57.162780 (+ 468us) server_negotiation.cc:407] Connection header received
0810 18:52:57.162964 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:57.162967 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:57.163014 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:57.163066 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:57.163570 (+ 504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46918: BlockingRecv error: recv got EOF from 127.0.0.1:46918 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260810 18:52:57.169519 12367 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:52:57.167991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46930 (local address 127.18.168.189:33023)
0810 18:52:57.168088 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:57.168092 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:57.168554 (+ 462us) server_negotiation.cc:407] Connection header received
0810 18:52:57.168799 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:57.168805 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:57.168870 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:57.168923 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:57.169413 (+ 490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46930: BlockingRecv error: recv got EOF from 127.0.0.1:46930 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260810 18:52:57.178409 12367 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:52:57.175822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46940 (local address 127.18.168.189:33023)
0810 18:52:57.175961 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:57.175965 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:57.176440 (+ 475us) server_negotiation.cc:407] Connection header received
0810 18:52:57.176648 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:57.176651 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:57.176727 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:57.176780 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:57.178313 (+ 1533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46940: BlockingRecv error: recv got EOF from 127.0.0.1:46940 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260810 18:52:57.191025 12367 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:52:57.189078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46954 (local address 127.18.168.189:33023)
0810 18:52:57.189195 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:57.189201 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:57.189792 (+ 591us) server_negotiation.cc:407] Connection header received
0810 18:52:57.190010 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:57.190014 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:57.190215 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:57.190280 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:57.190902 (+ 622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46954: BlockingRecv error: recv got EOF from 127.0.0.1:46954 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260810 18:52:57.212033 12367 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:52:57.209975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46964 (local address 127.18.168.189:33023)
0810 18:52:57.210111 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:57.210116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:57.210758 (+ 642us) server_negotiation.cc:407] Connection header received
0810 18:52:57.211079 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:57.211083 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:57.211149 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:57.211227 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:57.211906 (+ 679us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46964: BlockingRecv error: recv got EOF from 127.0.0.1:46964 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260810 18:52:57.248579 12367 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:52:57.247228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46980 (local address 127.18.168.189:33023)
0810 18:52:57.247370 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:57.247375 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:57.247559 (+ 184us) server_negotiation.cc:407] Connection header received
0810 18:52:57.247693 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:57.247697 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:57.247760 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:57.247868 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:57.248441 (+ 573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46980: BlockingRecv error: recv got EOF from 127.0.0.1:46980 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260810 18:52:57.315833 12367 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:52:57.314684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46986 (local address 127.18.168.189:33023)
0810 18:52:57.314890 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:57.314894 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:57.314976 (+ 82us) server_negotiation.cc:407] Connection header received
0810 18:52:57.315103 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:57.315106 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:57.315155 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:57.315255 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:57.315711 (+ 456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46986: BlockingRecv error: recv got EOF from 127.0.0.1:46986 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
W20260810 18:52:57.447216 12367 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:52:57.445931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46994 (local address 127.18.168.189:33023)
0810 18:52:57.446073 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:57.446077 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:57.446329 (+ 252us) server_negotiation.cc:407] Connection header received
0810 18:52:57.446491 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:57.446494 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:57.446559 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:57.446663 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:57.447060 (+ 397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46994: BlockingRecv error: recv got EOF from 127.0.0.1:46994 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260810 18:52:57.706815 12367 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:52:57.705645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47010 (local address 127.18.168.189:33023)
0810 18:52:57.705804 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:57.705808 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:57.705922 (+ 114us) server_negotiation.cc:407] Connection header received
0810 18:52:57.706067 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:57.706070 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:57.706152 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:57.706230 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:57.706671 (+ 441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47010: BlockingRecv error: recv got EOF from 127.0.0.1:47010 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 18:52:58.032830 12487 catalog_manager.cc:1259] Loaded cluster ID: 13d5570948a84cb1842be1f01b48e870
I20260810 18:52:58.032907 12487 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 04c60618ac8f4866bb9125f6c6c1c98d: loading cluster ID for follower catalog manager: success
I20260810 18:52:58.034628 12487 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 04c60618ac8f4866bb9125f6c6c1c98d: acquiring CA information for follower catalog manager: success
I20260810 18:52:58.035841 12487 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 04c60618ac8f4866bb9125f6c6c1c98d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 18:52:58.073056 12504 catalog_manager.cc:1259] Loaded cluster ID: 13d5570948a84cb1842be1f01b48e870
I20260810 18:52:58.073133 12504 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 90a8c9bc1fea49cfb4ca1ed1e9eff9c6: loading cluster ID for follower catalog manager: success
I20260810 18:52:58.074786 12504 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 90a8c9bc1fea49cfb4ca1ed1e9eff9c6: acquiring CA information for follower catalog manager: success
I20260810 18:52:58.075899 12504 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 90a8c9bc1fea49cfb4ca1ed1e9eff9c6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 18:52:58.225680 12590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:58.221448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47012 (local address 127.18.168.189:33023)
0810 18:52:58.221744 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:58.221748 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:58.221870 (+ 122us) server_negotiation.cc:407] Connection header received
0810 18:52:58.222011 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:58.222014 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:58.222165 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:58.222278 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:52:58.222639 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:58.223158 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:58.224108 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:58.224330 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:58.224408 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:58.224728 (+ 320us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:52:58.225232 (+ 504us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:52:58.225380 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:58.225417 (+ 37us) server_negotiation.cc:299] Negotiation successful
0810 18:52:58.225467 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":101,"threads_started":1}
I20260810 18:52:58.232089 12597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:58.228914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45064 (local address 127.18.168.188:46205)
0810 18:52:58.229145 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:58.229148 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:58.229163 (+ 15us) server_negotiation.cc:407] Connection header received
0810 18:52:58.229239 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:58.229241 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:58.229346 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:58.229440 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:52:58.229581 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:58.230055 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:58.230817 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:58.231060 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:58.231144 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:58.231381 (+ 237us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:52:58.231745 (+ 364us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:52:58.231844 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:58.231897 (+ 53us) server_negotiation.cc:299] Negotiation successful
0810 18:52:58.231962 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":68,"threads_started":1}
I20260810 18:52:58.238575 12597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:58.235230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45078 (local address 127.18.168.188:46205)
0810 18:52:58.235399 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:58.235406 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:58.235471 (+ 65us) server_negotiation.cc:407] Connection header received
0810 18:52:58.235522 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:58.235528 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:58.235587 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:58.235656 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:52:58.235947 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:58.236566 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:58.237525 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:58.237730 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:58.237834 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:58.238094 (+ 260us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:52:58.238158 (+ 64us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:52:58.238238 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:58.238353 (+ 115us) server_negotiation.cc:299] Negotiation successful
0810 18:52:58.238404 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 18:52:58.239949 12590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:58.235369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47024 (local address 127.18.168.189:33023)
0810 18:52:58.235838 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:58.235843 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:58.236083 (+ 240us) server_negotiation.cc:407] Connection header received
0810 18:52:58.236133 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:58.236135 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:58.236190 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:58.236271 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:52:58.236521 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:58.237377 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:58.238665 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:58.239016 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:58.239157 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:58.239414 (+ 257us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:52:58.239498 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:52:58.239592 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:58.239719 (+ 127us) server_negotiation.cc:299] Negotiation successful
0810 18:52:58.239797 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389}
I20260810 18:52:58.240747 12606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:58.235566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42476 (local address 127.18.168.190:43139)
0810 18:52:58.235887 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:58.235906 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:58.235934 (+ 28us) server_negotiation.cc:407] Connection header received
0810 18:52:58.236034 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:58.236038 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:58.236202 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:58.236668 (+ 466us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:52:58.237064 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:58.237954 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:58.239201 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:58.239514 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:58.239620 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:58.239922 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:52:58.240275 (+ 353us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:52:58.240375 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:58.240508 (+ 133us) server_negotiation.cc:299] Negotiation successful
0810 18:52:58.240564 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":119,"threads_started":1}
I20260810 18:52:58.244067 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 12197
I20260810 18:52:58.254144 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 12281
I20260810 18:52:58.262794 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 12369
2026-08-10T18:52:58Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3159 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:52:58 dist-test-slave-cbsf krb5kdc[12609](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:52:58 dist-test-slave-cbsf krb5kdc[12609](info): set up 2 sockets
Aug 10 18:52:58 dist-test-slave-cbsf krb5kdc[12609](info): commencing operation
krb5kdc: starting...
W20260810 18:52:59.370447 19106 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/krb5kdc/test-user.keytab.
Aug 10 18:52:59 dist-test-slave-cbsf krb5kdc[12609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387979, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:52:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:52:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.168.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
Entry for principal kudu/127.18.168.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab.
I20260810 18:52:59.494246 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:36165
--webserver_interface=127.18.168.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:34031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:36165,127.18.168.189:45453,127.18.168.188:34063
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:52:59.605494 12625 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 18:52:59.605738 12625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:59.605785 12625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:59.605820 12625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:59.610046 12625 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:52:59.610109 12625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:59.610129 12625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:59.610146 12625 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:52:59.610162 12625 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:52:59.615465 12625 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:34031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:36165,127.18.168.189:45453,127.18.168.188:34063
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.190.keytab
--principal=kudu/127.18.168.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.190:36165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.168.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12625
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:59.616456 12625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:59.617496 12625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:59.625604 12631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:59.625612 12633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:59.625674 12625 server_base.cc:1034] running on GCE node
W20260810 18:52:59.625653 12630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:59.626359 12625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:59.627338 12625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:59.628496 12625 hybrid_clock.cc:648] HybridClock initialized: now 1786387979628476 us; error 30 us; skew 500 ppm
Aug 10 18:52:59 dist-test-slave-cbsf krb5kdc[12609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387979, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:59.631464 12625 init.cc:377] Logged in from keytab as kudu/127.18.168.190@KRBTEST.COM (short username kudu)
I20260810 18:52:59.632299 12625 webserver.cc:460] Webserver started at http://127.18.168.190:40003/ using document root <none> and password file <none>
I20260810 18:52:59.632845 12625 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:59.632910 12625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:59.633077 12625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:59.634964 12625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-0/data/instance:
uuid: "b2615384213245ecbc7c452326c2872a"
format_stamp: "Formatted at 2026-08-10 18:52:59 on dist-test-slave-cbsf"
server_key: "c814c958acc18ebaa521d57b2b161e44"
server_key_iv: "8155b9e09d650364ac687f623c5a5fa1"
server_key_version: "encryptionkey@0"
I20260810 18:52:59.635480 12625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-0/wal/instance:
uuid: "b2615384213245ecbc7c452326c2872a"
format_stamp: "Formatted at 2026-08-10 18:52:59 on dist-test-slave-cbsf"
server_key: "c814c958acc18ebaa521d57b2b161e44"
server_key_iv: "8155b9e09d650364ac687f623c5a5fa1"
server_key_version: "encryptionkey@0"
I20260810 18:52:59.638917 12625 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 18:52:59.641151 12640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:59.642393 12625 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:52:59.642525 12625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-0/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-0/wal
uuid: "b2615384213245ecbc7c452326c2872a"
format_stamp: "Formatted at 2026-08-10 18:52:59 on dist-test-slave-cbsf"
server_key: "c814c958acc18ebaa521d57b2b161e44"
server_key_iv: "8155b9e09d650364ac687f623c5a5fa1"
server_key_version: "encryptionkey@0"
I20260810 18:52:59.642632 12625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:59.657547 12625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:59.660199 12625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:59.660391 12625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:59.667500 12625 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.190:36165
I20260810 18:52:59.667798 12697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.190:36165 every 8 connection(s)
I20260810 18:52:59.668424 12625 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-0/data/info.pb
I20260810 18:52:59.671146 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 12625
I20260810 18:52:59.671271 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-0/wal/instance
I20260810 18:52:59.671274 12698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:59.671625 19106 external_mini_cluster.cc:1428] Setting key e23ee37286eba4908f0bff51013c346e
I20260810 18:52:59.675776 12698 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.168.190" port: 36165 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.168.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.189@KRBTEST.COM" created.
Aug 10 18:52:59 dist-test-slave-cbsf krb5kdc[12609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387979, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:52:59.692541 12703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:59.678283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:36165 (local address 127.0.0.1:50980)
0810 18:52:59.678778 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:59.678819 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:59.679748 (+ 929us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:59.680415 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:59.680421 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:59.680874 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:52:59.681122 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:59.681136 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:59.682447 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:59.682450 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:59.683301 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:59.683311 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:59.684032 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:59.684036 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:59.684165 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:59.684814 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:52:59.684831 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:52:59.686773 (+ 1942us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:52:59.689289 (+ 2516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:59.689303 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:59.689313 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:59.689550 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:59.690001 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:59.690003 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:59.690005 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:59.690124 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:59.691002 (+ 878us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:52:59.691007 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:52:59.691145 (+ 138us) client_negotiation.cc:769] Sending connection context
0810 18:52:59.691295 (+ 150us) client_negotiation.cc:240] Negotiation successful
0810 18:52:59.691584 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":145,"threads_started":1}
I20260810 18:52:59.692544 12704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:59.679274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50980 (local address 127.18.168.190:36165)
0810 18:52:59.679577 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:59.679599 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:59.679643 (+ 44us) server_negotiation.cc:407] Connection header received
0810 18:52:59.680044 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:59.680048 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:59.680232 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:59.680416 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:59.681277 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:59.682318 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:59.683421 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:59.683878 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:59.684052 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:59.686942 (+ 2890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:59.686948 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:59.686959 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:59.686981 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:59.689148 (+ 2167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:59.689691 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:59.689702 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:59.689706 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:59.689887 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:59.690226 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:59.690231 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:59.690233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:59.690876 (+ 643us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:59.691039 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:59.691422 (+ 383us) server_negotiation.cc:299] Negotiation successful
0810 18:52:59.691727 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":81,"server-negotiator.queue_time_us":195,"thread_start_us":95,"threads_started":1}
I20260810 18:52:59.695390 12698 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.168.189" port: 45453 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:52:59.696373 12703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:59.696064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.189:45453 (local address 127.0.0.1:36354)
0810 18:52:59.696230 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:59.696307 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.189:45453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260810 18:52:59.696595 12642 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.168.189:45453: connect: Connection refused (error 111)
I20260810 18:52:59.697685 12703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:59.697378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.189:45453 (local address 127.0.0.1:36360)
0810 18:52:59.697507 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:59.697599 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.189:45453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260810 18:52:59.697893 12698 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.168.189:45453: Network error: Client connection negotiation failed: client connection to 127.18.168.189:45453: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.189.keytab.
Entry for principal kudu/127.18.168.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.189.keytab.
I20260810 18:52:59.703271 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.189:45453
--webserver_interface=127.18.168.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.189.keytab
--principal=kudu/127.18.168.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:34031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:36165,127.18.168.189:45453,127.18.168.188:34063
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:52:59.747141 12698 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.168.189" port: 45453 } attempt: 1
I20260810 18:52:59.748020 12703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:59.747704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.189:45453 (local address 127.0.0.1:36374)
0810 18:52:59.747852 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:59.747939 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.189:45453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 18:52:59.748811 12703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:59.748530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.189:45453 (local address 127.0.0.1:36384)
0810 18:52:59.748649 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:59.748746 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.189:45453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260810 18:52:59.748955 12698 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.168.189:45453: Network error: Client connection negotiation failed: client connection to 127.18.168.189:45453: connect: Connection refused (error 111)
I20260810 18:52:59.817121 12698 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.168.189" port: 45453 } attempt: 2
I20260810 18:52:59.818035 12703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:59.817716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.189:45453 (local address 127.0.0.1:36392)
0810 18:52:59.817873 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:59.817960 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.189:45453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260810 18:52:59.818811 12703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:59.818605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.189:45453 (local address 127.0.0.1:36394)
0810 18:52:59.818716 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:59.818763 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.189:45453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260810 18:52:59.818969 12698 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.168.189:45453: Network error: Client connection negotiation failed: client connection to 127.18.168.189:45453: connect: Connection refused (error 111)
W20260810 18:52:59.820086 12707 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 18:52:59.820333 12707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:59.820403 12707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:59.820508 12707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:59.824646 12707 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:52:59.824746 12707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:59.824802 12707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:59.824839 12707 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:52:59.824889 12707 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:52:59.830101 12707 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:34031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:36165,127.18.168.189:45453,127.18.168.188:34063
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.189.keytab
--principal=kudu/127.18.168.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.189:45453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.168.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12707
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:52:59.831128 12707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:59.832135 12707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:59.840436 12713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:59.840412 12712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:59.840636 12715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:59.840853 12707 server_base.cc:1034] running on GCE node
I20260810 18:52:59.841329 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:59.842330 12707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:59.843511 12707 hybrid_clock.cc:648] HybridClock initialized: now 1786387979843499 us; error 30 us; skew 500 ppm
Aug 10 18:52:59 dist-test-slave-cbsf krb5kdc[12609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387979, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:59.846242 12707 init.cc:377] Logged in from keytab as kudu/127.18.168.189@KRBTEST.COM (short username kudu)
I20260810 18:52:59.847091 12707 webserver.cc:460] Webserver started at http://127.18.168.189:43831/ using document root <none> and password file <none>
I20260810 18:52:59.847644 12707 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:59.847731 12707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:59.847958 12707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:59.850101 12707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-1/data/instance:
uuid: "d684a82e5f7f43eeb3864a55d96c0418"
format_stamp: "Formatted at 2026-08-10 18:52:59 on dist-test-slave-cbsf"
server_key: "72d9a653b49bee02df5f8e39dd9f6320"
server_key_iv: "b544ef6be19f23a8b1f49478d5476736"
server_key_version: "encryptionkey@0"
I20260810 18:52:59.850652 12707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-1/wal/instance:
uuid: "d684a82e5f7f43eeb3864a55d96c0418"
format_stamp: "Formatted at 2026-08-10 18:52:59 on dist-test-slave-cbsf"
server_key: "72d9a653b49bee02df5f8e39dd9f6320"
server_key_iv: "b544ef6be19f23a8b1f49478d5476736"
server_key_version: "encryptionkey@0"
I20260810 18:52:59.854432 12707 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260810 18:52:59.856832 12723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:59.858028 12707 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 18:52:59.858178 12707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-1/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-1/wal
uuid: "d684a82e5f7f43eeb3864a55d96c0418"
format_stamp: "Formatted at 2026-08-10 18:52:59 on dist-test-slave-cbsf"
server_key: "72d9a653b49bee02df5f8e39dd9f6320"
server_key_iv: "b544ef6be19f23a8b1f49478d5476736"
server_key_version: "encryptionkey@0"
I20260810 18:52:59.858286 12707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:59.879729 12707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:59.882649 12707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:59.882851 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:59.890101 12707 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.189:45453
I20260810 18:52:59.890385 12780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.189:45453 every 8 connection(s)
I20260810 18:52:59.891049 12707 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-1/data/info.pb
I20260810 18:52:59.893770 12781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:59.897653 12781 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.168.190" port: 36165 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:52:59.901202 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 12707
I20260810 18:52:59.901333 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-1/wal/instance
I20260810 18:52:59.901726 19106 external_mini_cluster.cc:1428] Setting key 58f38c799eb1c428f575a413f7b5490a
Aug 10 18:52:59 dist-test-slave-cbsf krb5kdc[12609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387979, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.189@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
I20260810 18:52:59.910151 12698 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.168.189" port: 45453 } attempt: 3
WARNING: no policy specified for kudu/127.18.168.188@KRBTEST.COM; defaulting to no policy
I20260810 18:52:59.913882 12783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:59.899773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:36165 (local address 127.0.0.1:50990)
0810 18:52:59.900203 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:59.900245 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:59.901116 (+ 871us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:59.901701 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:59.901708 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:59.902108 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:52:59.902565 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:59.902578 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:59.903716 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:59.903722 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:59.905217 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:59.905233 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:59.905908 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:59.905911 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:59.906052 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:59.906659 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:52:59.906676 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:52:59.908176 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:52:59.910283 (+ 2107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:59.910289 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:59.910303 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:59.910650 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:59.911015 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:59.911018 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:59.911020 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:59.911193 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:59.911739 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:52:59.911747 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:52:59.912072 (+ 325us) client_negotiation.cc:769] Sending connection context
0810 18:52:59.912314 (+ 242us) client_negotiation.cc:240] Negotiation successful
0810 18:52:59.912876 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":178,"thread_start_us":107,"threads_started":1}
I20260810 18:52:59.914597 12704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:59.899462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50990 (local address 127.18.168.190:36165)
0810 18:52:59.899603 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:59.899607 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:59.900355 (+ 748us) server_negotiation.cc:407] Connection header received
0810 18:52:59.901339 (+ 984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:59.901345 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:59.901410 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:59.901506 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:59.902721 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:59.903545 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:59.905351 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:59.905759 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:59.905888 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:59.908357 (+ 2469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:59.908364 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:59.908369 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:59.908407 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:59.910155 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:59.910777 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:59.910781 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:59.910783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:59.910888 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:59.911329 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:59.911335 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:59.911337 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:59.911576 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:59.911674 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:59.914284 (+ 2610us) server_negotiation.cc:299] Negotiation successful
0810 18:52:59.914448 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.168.188@KRBTEST.COM" created.
I20260810 18:52:59.915860 12781 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.168.189" port: 45453 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:52:59 dist-test-slave-cbsf krb5kdc[12609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387979, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for kudu/127.18.168.189@KRBTEST.COM
I20260810 18:52:59.921039 12703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:59.910632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.189:45453 (local address 127.0.0.1:36400)
0810 18:52:59.910835 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:59.910850 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:59.910930 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:59.911668 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:59.911671 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:59.911842 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:52:59.911956 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:59.911962 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:59.913008 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:59.913011 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:59.913783 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:59.913794 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:59.914573 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:59.914576 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:59.914691 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:59.915294 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:52:59.915305 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:52:59.916848 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:52:59.919145 (+ 2297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:59.919152 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:59.919157 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:59.919459 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:59.919799 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:59.919802 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:59.919804 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:59.919892 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:59.920468 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:52:59.920472 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:52:59.920578 (+ 106us) client_negotiation.cc:769] Sending connection context
0810 18:52:59.920732 (+ 154us) client_negotiation.cc:240] Negotiation successful
0810 18:52:59.920876 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115}
I20260810 18:52:59.921482 12785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:59.910775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36400 (local address 127.18.168.189:45453)
0810 18:52:59.911221 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:59.911236 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:59.911262 (+ 26us) server_negotiation.cc:407] Connection header received
0810 18:52:59.911305 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:59.911313 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:59.911493 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:59.911647 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:59.912070 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:59.912882 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:59.913926 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:59.914440 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:59.914572 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:59.916973 (+ 2401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:59.916986 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:59.917002 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:59.917031 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:59.919014 (+ 1983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:59.919575 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:59.919582 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:59.919583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:59.919680 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:59.919986 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:59.919988 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:59.919989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:59.920344 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:59.920505 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:59.920858 (+ 353us) server_negotiation.cc:299] Negotiation successful
0810 18:52:59.920997 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"spinlock_wait_cycles":7040,"thread_start_us":122,"threads_started":1}
Aug 10 18:52:59 dist-test-slave-cbsf krb5kdc[12609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387979, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.189@KRBTEST.COM for kudu/127.18.168.189@KRBTEST.COM
I20260810 18:52:59.923830 12698 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.168.188" port: 34063 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:52:59.924537 12703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:59.924294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:34063 (local address 127.0.0.1:60372)
0810 18:52:59.924409 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:59.924462 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.188:34063: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 18:52:59.925335 12703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:59.925030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:34063 (local address 127.0.0.1:60374)
0810 18:52:59.925208 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:59.925268 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.188:34063: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
W20260810 18:52:59.925472 12698 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.168.188:34063: Network error: Client connection negotiation failed: client connection to 127.18.168.188:34063: connect: Connection refused (error 111)
I20260810 18:52:59.925825 12783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:59.916420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.189:45453 (local address 127.0.0.1:36406)
0810 18:52:59.916635 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:59.916652 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:59.916757 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:59.917019 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:59.917022 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:59.917197 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:52:59.917294 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:59.917299 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:59.918423 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:59.918428 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:59.919225 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:59.919233 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:59.919720 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:59.919725 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:59.919867 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:59.920336 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:52:59.920356 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:52:59.922059 (+ 1703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:52:59.924151 (+ 2092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:59.924155 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:59.924157 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:59.924410 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:59.924749 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:59.924752 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:59.924754 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:59.924843 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:59.925289 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:52:59.925292 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:52:59.925508 (+ 216us) client_negotiation.cc:769] Sending connection context
0810 18:52:59.925619 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 18:52:59.925724 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"spinlock_wait_cycles":768}
I20260810 18:52:59.926047 12786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:59.916428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36406 (local address 127.18.168.189:45453)
0810 18:52:59.916745 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:59.916755 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:59.916772 (+ 17us) server_negotiation.cc:407] Connection header received
0810 18:52:59.916865 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:59.916868 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:59.916922 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:59.917009 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:59.917420 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:59.918294 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:59.919333 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:59.919595 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:59.919699 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:59.922232 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:59.922236 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:59.922239 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:59.922262 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:59.924031 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:59.924513 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:59.924517 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:59.924518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:59.924622 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:59.924951 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:59.924954 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:59.924955 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:59.925185 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:59.925288 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:59.925685 (+ 397us) server_negotiation.cc:299] Negotiation successful
0810 18:52:59.925779 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":93,"threads_started":1}
I20260810 18:52:59.926637 12781 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.168.188" port: 34063 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:52:59.927286 12783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:59.927031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:34063 (local address 127.0.0.1:60386)
0810 18:52:59.927172 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:59.927238 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.188:34063: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 18:52:59.927491 12724 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.168.188:34063: connect: Connection refused (error 111)
I20260810 18:52:59.928340 12783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:59.928096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:34063 (local address 127.0.0.1:60402)
0810 18:52:59.928189 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:59.928256 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.188:34063: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260810 18:52:59.928473 12781 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.168.188:34063: Network error: Client connection negotiation failed: client connection to 127.18.168.188:34063: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.168.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.188.keytab.
Entry for principal kudu/127.18.168.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.188.keytab.
I20260810 18:52:59.931582 19106 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1VMCzb/build/debug/bin/kudu
/tmp/dist-test-task1VMCzb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.188:34063
--webserver_interface=127.18.168.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.188.keytab
--principal=kudu/127.18.168.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.168.148:34031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.168.190:36165,127.18.168.189:45453,127.18.168.188:34063
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:52:59.956647 12698 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.168.188" port: 34063 } attempt: 1
I20260810 18:52:59.957495 12703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:59.957215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:34063 (local address 127.0.0.1:60414)
0810 18:52:59.957363 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:59.957420 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.188:34063: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260810 18:52:59.958220 12703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:59.957998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:34063 (local address 127.0.0.1:60428)
0810 18:52:59.958119 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:59.958167 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.188:34063: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260810 18:52:59.958351 12698 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.168.188:34063: Network error: Client connection negotiation failed: client connection to 127.18.168.188:34063: connect: Connection refused (error 111)
I20260810 18:52:59.977720 12781 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.168.188" port: 34063 } attempt: 1
I20260810 18:52:59.978551 12783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:59.978269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:34063 (local address 127.0.0.1:60444)
0810 18:52:59.978416 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:59.978476 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.188:34063: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260810 18:52:59.979285 12783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:59.979062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:34063 (local address 127.0.0.1:60446)
0810 18:52:59.979184 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:59.979235 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.188:34063: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260810 18:52:59.979417 12781 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.168.188:34063: Network error: Client connection negotiation failed: client connection to 127.18.168.188:34063: connect: Connection refused (error 111)
I20260810 18:53:00.033511 12698 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.168.188" port: 34063 } attempt: 2
I20260810 18:53:00.034433 12703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:00.034119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:34063 (local address 127.0.0.1:60456)
0810 18:53:00.034292 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:00.034355 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.188:34063: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260810 18:53:00.035166 12703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:00.034950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:34063 (local address 127.0.0.1:60462)
0810 18:53:00.035058 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:00.035111 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.188:34063: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260810 18:53:00.035301 12698 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.168.188:34063: Network error: Client connection negotiation failed: client connection to 127.18.168.188:34063: connect: Connection refused (error 111)
W20260810 18:53:00.045816 12789 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 18:53:00.046135 12789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:00.046205 12789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:00.046314 12789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260810 18:53:00.047565 12781 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.168.188" port: 34063 } attempt: 2
I20260810 18:53:00.048273 12783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:00.047994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:34063 (local address 127.0.0.1:60472)
0810 18:53:00.048140 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:00.048204 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.188:34063: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 18:53:00.049038 12783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:00.048832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:34063 (local address 127.0.0.1:60476)
0810 18:53:00.048935 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:00.048982 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.168.188:34063: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 18:53:00.049165 12781 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.168.188:34063: Network error: Client connection negotiation failed: client connection to 127.18.168.188:34063: connect: Connection refused (error 111)
W20260810 18:53:00.050980 12789 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:53:00.051069 12789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:00.051148 12789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:00.051211 12789 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:53:00.051252 12789 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:53:00.056560 12789 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.168.148:34031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.168.190:36165,127.18.168.189:45453,127.18.168.188:34063
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/krb5kdc/kudu_127.18.168.188.keytab
--principal=kudu/127.18.168.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.168.188:34063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.168.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:00.057674 12789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:00.058756 12789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:00.067323 12795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:00.067330 12794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:00.067317 12797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:00.067883 12789 server_base.cc:1034] running on GCE node
I20260810 18:53:00.068476 12789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:00.069619 12789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:00.070842 12789 hybrid_clock.cc:648] HybridClock initialized: now 1786387980070820 us; error 32 us; skew 500 ppm
Aug 10 18:53:00 dist-test-slave-cbsf krb5kdc[12609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387980, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:00.074223 12789 init.cc:377] Logged in from keytab as kudu/127.18.168.188@KRBTEST.COM (short username kudu)
I20260810 18:53:00.075277 12789 webserver.cc:460] Webserver started at http://127.18.168.188:41727/ using document root <none> and password file <none>
I20260810 18:53:00.075965 12789 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:00.076064 12789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:00.076282 12789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:00.078392 12789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-2/data/instance:
uuid: "22bd4a7709224b68a73df6406aa7c8df"
format_stamp: "Formatted at 2026-08-10 18:53:00 on dist-test-slave-cbsf"
server_key: "8c709bf2497b7dc62cb61d28c12daf81"
server_key_iv: "be14c29e1c9d7cd89da559dca114b567"
server_key_version: "encryptionkey@0"
I20260810 18:53:00.078971 12789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-2/wal/instance:
uuid: "22bd4a7709224b68a73df6406aa7c8df"
format_stamp: "Formatted at 2026-08-10 18:53:00 on dist-test-slave-cbsf"
server_key: "8c709bf2497b7dc62cb61d28c12daf81"
server_key_iv: "be14c29e1c9d7cd89da559dca114b567"
server_key_version: "encryptionkey@0"
I20260810 18:53:00.082702 12789 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260810 18:53:00.085078 12805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:00.086239 12789 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:53:00.086378 12789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-2/data,/tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-2/wal
uuid: "22bd4a7709224b68a73df6406aa7c8df"
format_stamp: "Formatted at 2026-08-10 18:53:00 on dist-test-slave-cbsf"
server_key: "8c709bf2497b7dc62cb61d28c12daf81"
server_key_iv: "be14c29e1c9d7cd89da559dca114b567"
server_key_version: "encryptionkey@0"
I20260810 18:53:00.086484 12789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:00.098488 12789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:00.101649 12789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:00.101840 12789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:00.109052 12789 rpc_server.cc:307] RPC server started. Bound to: 127.18.168.188:34063
I20260810 18:53:00.109354 12860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.168.188:34063 every 8 connection(s)
I20260810 18:53:00.110028 12789 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-2/data/info.pb
I20260810 18:53:00.112776 12861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:00.116513 12861 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.168.190" port: 36165 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:53:00.117834 19106 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1VMCzb/build/debug/bin/kudu as pid 12789
I20260810 18:53:00.117972 19106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1VMCzb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387687857264-19106-0/minicluster-data/master-2/wal/instance
I20260810 18:53:00.118286 19106 external_mini_cluster.cc:1428] Setting key a65ab1d8635157ec069c3702eb0785ab
Aug 10 18:53:00 dist-test-slave-cbsf krb5kdc[12609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387980, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.188@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
Aug 10 18:53:00 dist-test-slave-cbsf krb5kdc[12609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387979, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.168.188@KRBTEST.COM
I20260810 18:53:00.130457 12704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:00.118547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51002 (local address 127.18.168.190:36165)
0810 18:53:00.118719 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:00.118724 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:00.119682 (+ 958us) server_negotiation.cc:407] Connection header received
0810 18:53:00.120457 (+ 775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:00.120460 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:00.120506 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:00.120573 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:00.121515 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:00.122014 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:00.123141 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:00.123476 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:00.123556 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:00.126410 (+ 2854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:00.126413 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:00.126415 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:00.126438 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:00.128103 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:00.128632 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:00.128635 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:00.128636 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:00.128769 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:00.129152 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:00.129155 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:00.129156 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:00.129411 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:00.129514 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:00.130008 (+ 494us) server_negotiation.cc:299] Negotiation successful
0810 18:53:00.130155 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 18:53:00.131134 12864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:00.118929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.190:36165 (local address 127.0.0.1:51002)
0810 18:53:00.119525 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:00.119570 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:00.120245 (+ 675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:00.120660 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:00.120665 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:00.121113 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:00.121347 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:00.121359 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:00.122129 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:00.122132 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:00.123027 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:00.123036 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:00.123624 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:00.123628 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:00.123784 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:00.124405 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:00.124421 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:00.126239 (+ 1818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:00.128219 (+ 1980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:00.128230 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:00.128239 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:00.128529 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:00.128863 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:00.128869 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:00.128872 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:00.129036 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:00.129519 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:00.129524 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:00.129739 (+ 215us) client_negotiation.cc:769] Sending connection context
0810 18:53:00.129931 (+ 192us) client_negotiation.cc:240] Negotiation successful
0810 18:53:00.130239 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"mutex_wait_us":1,"spinlock_wait_cycles":2176,"thread_start_us":146,"threads_started":1}
I20260810 18:53:00.131132 12865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:00.118968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60488 (local address 127.18.168.188:34063)
0810 18:53:00.119388 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:00.119407 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:00.119448 (+ 41us) server_negotiation.cc:407] Connection header received
0810 18:53:00.120245 (+ 797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:00.120268 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:00.120513 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:00.120718 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:00.121442 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:00.122469 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:00.123396 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:00.123866 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:00.123982 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:00.126870 (+ 2888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:00.126875 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:00.126884 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:00.126905 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:00.128375 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:00.128907 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:00.128911 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:00.128913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:00.129013 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:00.129323 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:00.129329 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:00.129332 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:00.129744 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:00.129944 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:00.130201 (+ 257us) server_negotiation.cc:299] Negotiation successful
0810 18:53:00.130336 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"spinlock_wait_cycles":18048,"thread_start_us":169,"threads_started":1}
I20260810 18:53:00.132411 12861 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.168.189" port: 45453 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:53:00.134461 12698 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.168.188" port: 34063 } attempt: 3
Aug 10 18:53:00 dist-test-slave-cbsf krb5kdc[12609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387980, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.188@KRBTEST.COM for kudu/127.18.168.189@KRBTEST.COM
Aug 10 18:53:00 dist-test-slave-cbsf krb5kdc[12609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387979, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.168.190@KRBTEST.COM
Aug 10 18:53:00 dist-test-slave-cbsf krb5kdc[12609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387979, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.190@KRBTEST.COM for kudu/127.18.168.188@KRBTEST.COM
I20260810 18:53:00.140321 12781 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.168.188" port: 34063 } attempt: 3
I20260810 18:53:00.142619 12786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:00.133041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36418 (local address 127.18.168.189:45453)
0810 18:53:00.133187 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:00.133192 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:00.133207 (+ 15us) server_negotiation.cc:407] Connection header received
0810 18:53:00.133253 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:00.133256 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:00.133304 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:00.133388 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:00.133852 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:00.134463 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:00.135534 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:00.135790 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:00.135906 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:00.138298 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:00.138302 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:00.138305 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:00.138332 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:00.140019 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:00.140865 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:00.140868 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:00.140870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:00.141237 (+ 367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:00.141578 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:00.141582 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:00.141584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:00.141847 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:00.141942 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:00.142281 (+ 339us) server_negotiation.cc:299] Negotiation successful
0810 18:53:00.142414 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 18:53:00.143134 12864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:00.132894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.189:45453 (local address 127.0.0.1:36418)
0810 18:53:00.133068 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:00.133081 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:00.133164 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:00.133422 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:00.133427 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:00.133597 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:00.133721 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:00.133731 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:00.134571 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:00.134576 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:00.135391 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:00.135403 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:00.135905 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:00.135909 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:00.136008 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:00.136722 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:00.136738 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:00.138129 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:00.140182 (+ 2053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:00.140185 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:00.140187 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:00.140748 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:00.141374 (+ 626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:00.141380 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:00.141383 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:00.141487 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:00.141953 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:00.141958 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:00.142062 (+ 104us) client_negotiation.cc:769] Sending connection context
0810 18:53:00.142819 (+ 757us) client_negotiation.cc:240] Negotiation successful
0810 18:53:00.142957 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"spinlock_wait_cycles":3456}
I20260810 18:53:00.144245 12703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:00.134821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:34063 (local address 127.0.0.1:60490)
0810 18:53:00.134978 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:00.134990 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:00.135067 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:00.135309 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:00.135312 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:00.135485 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:00.135570 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:00.135577 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:00.136525 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:00.136528 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:00.137436 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:00.137447 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:00.137938 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:00.137942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:00.138037 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:00.138493 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:00.138508 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:00.140470 (+ 1962us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:00.142574 (+ 2104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:00.142577 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:00.142579 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:00.142856 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:00.143166 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:00.143172 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:00.143175 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:00.143266 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:00.143758 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:00.143761 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:00.143853 (+ 92us) client_negotiation.cc:769] Sending connection context
0810 18:53:00.143959 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 18:53:00.144088 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"mutex_wait_us":22}
I20260810 18:53:00.144351 12865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:00.134857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60490 (local address 127.18.168.188:34063)
0810 18:53:00.134948 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:00.134952 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:00.135031 (+ 79us) server_negotiation.cc:407] Connection header received
0810 18:53:00.135164 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:00.135167 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:00.135203 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:00.135293 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:00.135700 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:00.136426 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:00.137556 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:00.137790 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:00.137920 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:00.140655 (+ 2735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:00.140661 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:00.140667 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:00.140741 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:00.142459 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:00.142968 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:00.142971 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:00.142972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:00.143050 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:00.143369 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:00.143372 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:00.143373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:00.143584 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:00.143686 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:00.144047 (+ 361us) server_negotiation.cc:299] Negotiation successful
0810 18:53:00.144161 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35,"spinlock_wait_cycles":79104}
I20260810 18:53:00.144961 12861 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.168.188" port: 34063 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:53:00.146198 12704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:00.133619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51010 (local address 127.18.168.190:36165)
0810 18:53:00.133732 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:00.133737 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:00.133749 (+ 12us) server_negotiation.cc:407] Connection header received
0810 18:53:00.133855 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:00.133858 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:00.133894 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:00.133987 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:00.134642 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:00.135361 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:00.136418 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:00.136870 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:00.136984 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:00.140111 (+ 3127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:00.140118 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:00.140123 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:00.140154 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:00.142968 (+ 2814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:00.143841 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:00.143845 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:00.143846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:00.143923 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:00.144689 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:00.144700 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:00.144702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:00.144947 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:00.145220 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:00.145934 (+ 714us) server_negotiation.cc:299] Negotiation successful
0810 18:53:00.146049 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":109,"server-negotiator.queue_time_us":44}
Aug 10 18:53:00 dist-test-slave-cbsf krb5kdc[12609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387979, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.189@KRBTEST.COM for kudu/127.18.168.188@KRBTEST.COM
I20260810 18:53:00.147874 12698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2615384213245ecbc7c452326c2872a: Bootstrap starting.
I20260810 18:53:00.150765 12783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:00.140735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:34063 (local address 127.0.0.1:60500)
0810 18:53:00.140868 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:00.140884 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:00.140967 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:00.141915 (+ 948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:00.141917 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:00.142099 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:00.142209 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:00.142218 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:00.143787 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:00.143790 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:00.144589 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:00.144599 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:00.145257 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:00.145261 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:00.145363 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:00.145835 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:00.145847 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:00.147185 (+ 1338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:00.149187 (+ 2002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:00.149191 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:00.149193 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:00.149445 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:00.149745 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:00.149748 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:00.149750 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:00.149842 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:00.150302 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:00.150306 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:00.150404 (+ 98us) client_negotiation.cc:769] Sending connection context
0810 18:53:00.150492 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 18:53:00.150608 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260810 18:53:00.150942 12698 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2615384213245ecbc7c452326c2872a: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:00.151386 12866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:00.140976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60500 (local address 127.18.168.188:34063)
0810 18:53:00.141666 (+ 690us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:00.141676 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:00.141689 (+ 13us) server_negotiation.cc:407] Connection header received
0810 18:53:00.141722 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:00.141724 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:00.141767 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:00.142328 (+ 561us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:00.142430 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:00.143681 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:00.144745 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:00.145125 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:00.146336 (+ 1211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:00.147322 (+ 986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:00.147326 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:00.147328 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:00.147351 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:00.149057 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:00.149542 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:00.149545 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:00.149547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:00.149635 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:00.149961 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:00.149964 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:00.149965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:00.150196 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:00.150927 (+ 731us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:00.151009 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 18:53:00.151185 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":617,"thread_start_us":134,"threads_started":1}
I20260810 18:53:00.152154 12698 log.cc:824] T 00000000000000000000000000000000 P b2615384213245ecbc7c452326c2872a: Log is configured to *not* fsync() on all Append() calls
Aug 10 18:53:00 dist-test-slave-cbsf krb5kdc[12609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387980, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.168.188@KRBTEST.COM for kudu/127.18.168.188@KRBTEST.COM
I20260810 18:53:00.154510 12781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d684a82e5f7f43eeb3864a55d96c0418: Bootstrap starting.
I20260810 18:53:00.155368 12698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2615384213245ecbc7c452326c2872a: No bootstrap required, opened a new log
I20260810 18:53:00.157720 12781 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d684a82e5f7f43eeb3864a55d96c0418: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:00.158012 12864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:00.145397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.168.188:34063 (local address 127.0.0.1:60514)
0810 18:53:00.145598 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:00.145611 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:00.145687 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:00.149760 (+ 4073us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:00.149763 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:00.149930 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:00.150016 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:00.150022 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:00.150952 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:00.150955 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:00.151767 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:00.151780 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:00.152347 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:00.152353 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:00.152432 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:00.152885 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:00.152900 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:00.154301 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:00.156452 (+ 2151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:00.156455 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:00.156457 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:00.156792 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:00.157071 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:00.157075 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:00.157078 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:00.157147 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:00.157575 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:00.157577 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:00.157653 (+ 76us) client_negotiation.cc:769] Sending connection context
0810 18:53:00.157745 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 18:53:00.157863 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":17}
I20260810 18:53:00.158555 12865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:00.149383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60514 (local address 127.18.168.188:34063)
0810 18:53:00.149479 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:00.149482 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:00.149493 (+ 11us) server_negotiation.cc:407] Connection header received
0810 18:53:00.149529 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:00.149533 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:00.149570 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:00.149658 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:00.150121 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:00.150830 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:00.151882 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:00.152145 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:00.152220 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:00.154438 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:00.154442 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:00.154444 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:00.154467 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:00.156324 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:00.156901 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:00.156906 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:00.156909 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:00.156981 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:00.157253 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:00.157255 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:00.157257 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:00.157437 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:00.157542 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:00.158292 (+ 750us) server_negotiation.cc:299] Negotiation successful
0810 18:53:00.158408 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 18:53:00.158684 12781 log.cc:824] T 00000000000000000000000000000000 P d684a82e5f7f43eeb3864a55d96c0418: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:00.159546 12698 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2615384213245ecbc7c452326c2872a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2615384213245ecbc7c452326c2872a" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 36165 } } peers { permanent_uuid: "d684a82e5f7f43eeb3864a55d96c0418" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 45453 } } peers { permanent_uuid: "22bd4a7709224b68a73df6406aa7c8df" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 34063 } }
I20260810 18:53:00.159771 12698 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2615384213245ecbc7c452326c2872a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2615384213245ecbc7c452326c2872a, State: Initialized, Role: FOLLOWER
I20260810 18:53:00.160096 12698 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2615384213245ecbc7c452326c2872a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2615384213245ecbc7c452326c2872a" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 36165 } } peers { permanent_uuid: "d684a82e5f7f43eeb3864a55d96c0418" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 45453 } } peers { permanent_uuid: "22bd4a7709224b68a73df6406aa7c8df" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 34063 } }
Aug 10 18:53:00 dist-test-slave-cbsf krb5kdc[12609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387979, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.168.189@KRBTEST.COM
I20260810 18:53:00.160952 12781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d684a82e5f7f43eeb3864a55d96c0418: No bootstrap required, opened a new log
I20260810 18:53:00.161725 12698 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2615384213245ecbc7c452326c2872a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:00.161826 12861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22bd4a7709224b68a73df6406aa7c8df: Bootstrap starting.
W20260810 18:53:00.162009 12646 tablet.cc:2369] T 00000000000000000000000000000000 P b2615384213245ecbc7c452326c2872a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:53:00.164501 12861 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 22bd4a7709224b68a73df6406aa7c8df: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:00.165489 12861 log.cc:824] T 00000000000000000000000000000000 P 22bd4a7709224b68a73df6406aa7c8df: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:00.166008 12786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:00.150052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36428 (local address 127.18.168.189:45453)
0810 18:53:00.150159 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:00.150164 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:00.150328 (+ 164us) server_negotiation.cc:407] Connection header received
0810 18:53:00.150484 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:00.150487 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:00.150527 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:00.150571 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:00.152169 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:00.152938 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:00.157000 (+ 4062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:00.157332 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:00.157408 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:00.161623 (+ 4215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:00.161627 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:00.161629 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:00.161655 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:00.163831 (+ 2176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:00.164341 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:00.164345 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:00.164347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:00.164439 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:00.164889 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:00.164894 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:00.164895 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:00.165130 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:00.165285 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:00.165717 (+ 432us) server_negotiation.cc:299] Negotiation successful
0810 18:53:00.165849 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 18:53:00.166131 12781 raft_consensus.cc:348] T 00000000000000000000000000000000 P d684a82e5f7f43eeb3864a55d96c0418 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2615384213245ecbc7c452326c2872a" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 36165 } } peers { permanent_uuid: "d684a82e5f7f43eeb3864a55d96c0418" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 45453 } } peers { permanent_uuid: "22bd4a7709224b68a73df6406aa7c8df" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 34063 } }
I20260810 18:53:00.166338 12781 raft_consensus.cc:729] T 00000000000000000000000000000000 P d684a82e5f7f43eeb3864a55d96c0418 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d684a82e5f7f43eeb3864a55d96c0418, State: Initialized, Role: FOLLOWER
I20260810 18:53:00.166677 12781 consensus_queue.cc:260] T 00000000000000000000000000000000 P d684a82e5f7f43eeb3864a55d96c0418 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2615384213245ecbc7c452326c2872a" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 36165 } } peers { permanent_uuid: "d684a82e5f7f43eeb3864a55d96c0418" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 45453 } } peers { permanent_uuid: "22bd4a7709224b68a73df6406aa7c8df" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 34063 } }
I20260810 18:53:00.167541 12861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22bd4a7709224b68a73df6406aa7c8df: No bootstrap required, opened a new log
I20260810 18:53:00.167932 12781 sys_catalog.cc:564] T 00000000000000000000000000000000 P d684a82e5f7f43eeb3864a55d96c0418 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:00.168923 12870 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2615384213245ecbc7c452326c2872a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2615384213245ecbc7c452326c2872a" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 36165 } } peers { permanent_uuid: "d684a82e5f7f43eeb3864a55d96c0418" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 45453 } } peers { permanent_uuid: "22bd4a7709224b68a73df6406aa7c8df" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 34063 } } }
I20260810 18:53:00.169127 12870 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2615384213245ecbc7c452326c2872a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:53:00.166723 12625 master_runner.cc:186] Error getting master registration for 127.18.168.189:45453: OK, instance_id { permanent_uuid: "d684a82e5f7f43eeb3864a55d96c0418" instance_seqno: 1786387979888119 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 18:53:00.171017 12625 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 18:53:00.171656 12861 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22bd4a7709224b68a73df6406aa7c8df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2615384213245ecbc7c452326c2872a" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 36165 } } peers { permanent_uuid: "d684a82e5f7f43eeb3864a55d96c0418" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 45453 } } peers { permanent_uuid: "22bd4a7709224b68a73df6406aa7c8df" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 34063 } }
I20260810 18:53:00.171885 12861 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22bd4a7709224b68a73df6406aa7c8df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22bd4a7709224b68a73df6406aa7c8df, State: Initialized, Role: FOLLOWER
I20260810 18:53:00.172197 12861 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22bd4a7709224b68a73df6406aa7c8df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2615384213245ecbc7c452326c2872a" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 36165 } } peers { permanent_uuid: "d684a82e5f7f43eeb3864a55d96c0418" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 45453 } } peers { permanent_uuid: "22bd4a7709224b68a73df6406aa7c8df" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 34063 } }
I20260810 18:53:00.172492 19106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:53:00.173421 12861 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22bd4a7709224b68a73df6406aa7c8df [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:00.173687 12671 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b2615384213245ecbc7c452326c2872a"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51010
I20260810 18:53:00.173929 12671 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2615384213245ecbc7c452326c2872a [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 18:53:00.174072 12671 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2615384213245ecbc7c452326c2872a [term 0 FOLLOWER]: Advancing to term 1
W20260810 18:53:00.174290 12898 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d684a82e5f7f43eeb3864a55d96c0418: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:53:00.174424 12898 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:53:00.175343 12671 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2615384213245ecbc7c452326c2872a [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2615384213245ecbc7c452326c2872a" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 36165 } } peers { permanent_uuid: "d684a82e5f7f43eeb3864a55d96c0418" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 45453 } } peers { permanent_uuid: "22bd4a7709224b68a73df6406aa7c8df" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 34063 } }
I20260810 18:53:00.176370 12671 leader_election.cc:288] T 00000000000000000000000000000000 P b2615384213245ecbc7c452326c2872a [CANDIDATE]: Term 1 election: Requested vote from peers d684a82e5f7f43eeb3864a55d96c0418 (127.18.168.189:45453), 22bd4a7709224b68a73df6406aa7c8df (127.18.168.188:34063)
I20260810 18:53:00.176467 12900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22bd4a7709224b68a73df6406aa7c8df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2615384213245ecbc7c452326c2872a" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 36165 } } peers { permanent_uuid: "d684a82e5f7f43eeb3864a55d96c0418" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 45453 } } peers { permanent_uuid: "22bd4a7709224b68a73df6406aa7c8df" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 34063 } } }
I20260810 18:53:00.176966 12900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22bd4a7709224b68a73df6406aa7c8df [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:53:00.177115 12751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b2615384213245ecbc7c452326c2872a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d684a82e5f7f43eeb3864a55d96c0418"
I20260810 18:53:00.177351 12751 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d684a82e5f7f43eeb3864a55d96c0418 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:00.178710 12751 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d684a82e5f7f43eeb3864a55d96c0418 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2615384213245ecbc7c452326c2872a in term 1.
I20260810 18:53:00.175820 12707 master_runner.cc:186] Error getting master registration for 127.18.168.188:34063: OK, instance_id { permanent_uuid: "22bd4a7709224b68a73df6406aa7c8df" instance_seqno: 1786387980107038 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 18:53:00.179188 12642 leader_election.cc:302] T 00000000000000000000000000000000 P b2615384213245ecbc7c452326c2872a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2615384213245ecbc7c452326c2872a, d684a82e5f7f43eeb3864a55d96c0418; no voters:
I20260810 18:53:00.179294 12707 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260810 18:53:00.179440 12912 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 22bd4a7709224b68a73df6406aa7c8df: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260810 18:53:00.179483 12870 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2615384213245ecbc7c452326c2872a [term 1 FOLLOWER]: Leader election won for term 1
W20260810 18:53:00.179525 12912 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:53:00.179711 12870 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2615384213245ecbc7c452326c2872a [term 1 LEADER]: Becoming Leader. State: Replica: b2615384213245ecbc7c452326c2872a, State: Running, Role: LEADER
I20260810 18:53:00.180032 12870 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2615384213245ecbc7c452326c2872a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2615384213245ecbc7c452326c2872a" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 36165 } } peers { permanent_uuid: "d684a82e5f7f43eeb3864a55d96c0418" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 45453 } } peers { permanent_uuid: "22bd4a7709224b68a73df6406aa7c8df" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 34063 } }
I20260810 18:53:00.180145 12834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b2615384213245ecbc7c452326c2872a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "22bd4a7709224b68a73df6406aa7c8df"
I20260810 18:53:00.180338 12834 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22bd4a7709224b68a73df6406aa7c8df [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:00.181664 12834 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 22bd4a7709224b68a73df6406aa7c8df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2615384213245ecbc7c452326c2872a in term 1.
I20260810 18:53:00.183574 12914 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2615384213245ecbc7c452326c2872a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2615384213245ecbc7c452326c2872a. Latest consensus state: current_term: 1 leader_uuid: "b2615384213245ecbc7c452326c2872a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2615384213245ecbc7c452326c2872a" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 36165 } } peers { permanent_uuid: "d684a82e5f7f43eeb3864a55d96c0418" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 45453 } } peers { permanent_uuid: "22bd4a7709224b68a73df6406aa7c8df" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 34063 } } }
I20260810 18:53:00.186317 12914 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2615384213245ecbc7c452326c2872a [sys.catalog]: This master's current role is: LEADER
W20260810 18:53:00.185714 12883 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b2615384213245ecbc7c452326c2872a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:53:00.188498 12883 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:53:00.189064 12885 sys_catalog.cc:455] T 00000000000000000000000000000000 P d684a82e5f7f43eeb3864a55d96c0418 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2615384213245ecbc7c452326c2872a" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 36165 } } peers { permanent_uuid: "d684a82e5f7f43eeb3864a55d96c0418" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 45453 } } peers { permanent_uuid: "22bd4a7709224b68a73df6406aa7c8df" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 34063 } } }
I20260810 18:53:00.189881 12885 sys_catalog.cc:458] T 00000000000000000000000000000000 P d684a82e5f7f43eeb3864a55d96c0418 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:53:00.197192 12922 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:53:00.198149 12922 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:53:00.198388 12786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:00.186730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36440 (local address 127.18.168.189:45453)
0810 18:53:00.186866 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:00.186871 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:00.188865 (+ 1994us) server_negotiation.cc:407] Connection header received
0810 18:53:00.189079 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:00.189082 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:00.189131 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:00.189185 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:00.190271 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:00.191076 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:00.192274 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:00.192612 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:00.192710 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:00.194601 (+ 1891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:00.194605 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:00.194608 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:00.194632 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:00.196375 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:00.197008 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:00.197012 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:00.197014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:00.197104 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:00.197434 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:00.197438 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:00.197439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:00.197643 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:00.197780 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:00.198128 (+ 348us) server_negotiation.cc:299] Negotiation successful
0810 18:53:00.198238 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 18:53:00.204433 12834 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 22bd4a7709224b68a73df6406aa7c8df [term 1 FOLLOWER]: Refusing update from remote peer b2615384213245ecbc7c452326c2872a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:00.205119 12914 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b2615384213245ecbc7c452326c2872a [LEADER]: Connected to new peer: Peer: permanent_uuid: "22bd4a7709224b68a73df6406aa7c8df" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 34063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:00.209733 12900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22bd4a7709224b68a73df6406aa7c8df [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2615384213245ecbc7c452326c2872a. Latest consensus state: current_term: 1 leader_uuid: "b2615384213245ecbc7c452326c2872a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2615384213245ecbc7c452326c2872a" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 36165 } } peers { permanent_uuid: "d684a82e5f7f43eeb3864a55d96c0418" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 45453 } } peers { permanent_uuid: "22bd4a7709224b68a73df6406aa7c8df" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 34063 } } }
I20260810 18:53:00.209890 12900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22bd4a7709224b68a73df6406aa7c8df [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:53:00.210937 12751 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d684a82e5f7f43eeb3864a55d96c0418 [term 1 FOLLOWER]: Refusing update from remote peer b2615384213245ecbc7c452326c2872a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:00.211546 12914 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b2615384213245ecbc7c452326c2872a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d684a82e5f7f43eeb3864a55d96c0418" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 45453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:00.215415 12885 sys_catalog.cc:455] T 00000000000000000000000000000000 P d684a82e5f7f43eeb3864a55d96c0418 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2615384213245ecbc7c452326c2872a. Latest consensus state: current_term: 1 leader_uuid: "b2615384213245ecbc7c452326c2872a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2615384213245ecbc7c452326c2872a" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 36165 } } peers { permanent_uuid: "d684a82e5f7f43eeb3864a55d96c0418" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 45453 } } peers { permanent_uuid: "22bd4a7709224b68a73df6406aa7c8df" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 34063 } } }
I20260810 18:53:00.215549 12885 sys_catalog.cc:458] T 00000000000000000000000000000000 P d684a82e5f7f43eeb3864a55d96c0418 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:53:00.216907 12704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:00.188296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51012 (local address 127.18.168.190:36165)
0810 18:53:00.188406 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:00.188411 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:00.194405 (+ 5994us) server_negotiation.cc:407] Connection header received
0810 18:53:00.194603 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:00.194605 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:00.194655 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:00.194710 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:00.197788 (+ 3078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:00.198495 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:00.200356 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:00.200727 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:00.200802 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:00.211937 (+ 11135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:00.211941 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:00.211943 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:00.211972 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:00.213700 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:00.214740 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:00.214744 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:00.214746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:00.214849 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:00.215653 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:00.215656 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:00.215658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:00.215892 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:00.216001 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:00.216605 (+ 604us) server_negotiation.cc:299] Negotiation successful
0810 18:53:00.216764 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 18:53:00.218712 12914 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2615384213245ecbc7c452326c2872a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b2615384213245ecbc7c452326c2872a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2615384213245ecbc7c452326c2872a" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 36165 } } peers { permanent_uuid: "d684a82e5f7f43eeb3864a55d96c0418" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 45453 } } peers { permanent_uuid: "22bd4a7709224b68a73df6406aa7c8df" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 34063 } } }
I20260810 18:53:00.218832 12914 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2615384213245ecbc7c452326c2872a [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:00.219652 12865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:00.187558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60516 (local address 127.18.168.188:34063)
0810 18:53:00.187664 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:00.187668 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:00.192947 (+ 5279us) server_negotiation.cc:407] Connection header received
0810 18:53:00.193126 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:00.193132 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:00.193191 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:00.193246 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:00.195645 (+ 2399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:00.196426 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:00.199553 (+ 3127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:00.199864 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:00.199928 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:00.208660 (+ 8732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:00.208664 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:00.208667 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:00.208729 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:00.210552 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:00.214219 (+ 3667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:00.214223 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:00.214225 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:00.214316 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:00.215167 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:00.215173 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:00.215177 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:00.215424 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:00.215562 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:00.219378 (+ 3816us) server_negotiation.cc:299] Negotiation successful
0810 18:53:00.219514 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 18:53:00.219766 12900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22bd4a7709224b68a73df6406aa7c8df [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b2615384213245ecbc7c452326c2872a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2615384213245ecbc7c452326c2872a" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 36165 } } peers { permanent_uuid: "d684a82e5f7f43eeb3864a55d96c0418" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 45453 } } peers { permanent_uuid: "22bd4a7709224b68a73df6406aa7c8df" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 34063 } } }
I20260810 18:53:00.219869 12900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22bd4a7709224b68a73df6406aa7c8df [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:53:00.222820 12922 catalog_manager.cc:1347] Generated new cluster ID: 79112864ed444994b387422e3a593cc5
I20260810 18:53:00.222891 12922 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:53:00.226020 12870 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2615384213245ecbc7c452326c2872a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b2615384213245ecbc7c452326c2872a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2615384213245ecbc7c452326c2872a" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 36165 } } peers { permanent_uuid: "d684a82e5f7f43eeb3864a55d96c0418" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 45453 } } peers { permanent_uuid: "22bd4a7709224b68a73df6406aa7c8df" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 34063 } } }
I20260810 18:53:00.226130 12870 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2615384213245ecbc7c452326c2872a [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:00.229384 12928 mvcc.cc:205] Tried to move back new op lower bound from 7317045166909079552 to 7317045166820405248. Current Snapshot: MvccSnapshot[applied={T|T < 7317045166909079552}]
I20260810 18:53:00.229615 12930 mvcc.cc:205] Tried to move back new op lower bound from 7317045166909079552 to 7317045166820405248. Current Snapshot: MvccSnapshot[applied={T|T < 7317045166909079552}]
I20260810 18:53:00.229707 12885 sys_catalog.cc:455] T 00000000000000000000000000000000 P d684a82e5f7f43eeb3864a55d96c0418 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b2615384213245ecbc7c452326c2872a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2615384213245ecbc7c452326c2872a" member_type: VOTER last_known_addr { host: "127.18.168.190" port: 36165 } } peers { permanent_uuid: "d684a82e5f7f43eeb3864a55d96c0418" member_type: VOTER last_known_addr { host: "127.18.168.189" port: 45453 } } peers { permanent_uuid: "22bd4a7709224b68a73df6406aa7c8df" member_type: VOTER last_known_addr { host: "127.18.168.188" port: 34063 } } }
I20260810 18:53:00.229830 12885 sys_catalog.cc:458] T 00000000000000000000000000000000 P d684a82e5f7f43eeb3864a55d96c0418 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:53:00.254361 12922 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:53:00.255642 12922 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:53:00.279037 12922 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2615384213245ecbc7c452326c2872a: Generated new TSK 0
I20260810 18:53:00.279932 12922 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 18:53:00.307920 12704 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:53:00.305254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46420 (local address 127.18.168.190:36165)
0810 18:53:00.305392 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:00.305397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:00.306729 (+ 1332us) server_negotiation.cc:407] Connection header received
0810 18:53:00.306928 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:00.306931 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:00.306984 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:00.307052 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:00.307735 (+ 683us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46420: BlockingRecv error: recv got EOF from 127.0.0.1:46420 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260810 18:53:00.309326 12786 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:53:00.306121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51708 (local address 127.18.168.189:45453)
0810 18:53:00.306238 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:00.306243 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:00.308408 (+ 2165us) server_negotiation.cc:407] Connection header received
0810 18:53:00.308578 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:00.308582 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:00.308629 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:00.308678 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:00.309201 (+ 523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51708: BlockingRecv error: recv got EOF from 127.0.0.1:51708 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260810 18:53:00.311195 12865 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:53:00.309973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38706 (local address 127.18.168.188:34063)
0810 18:53:00.310098 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:00.310103 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:00.310315 (+ 212us) server_negotiation.cc:407] Connection header received
0810 18:53:00.310476 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:00.310479 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:00.310529 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:00.310580 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:00.311075 (+ 495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38706: BlockingRecv error: recv got EOF from 127.0.0.1:38706 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260810 18:53:00.315596 12786 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:53:00.314118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51716 (local address 127.18.168.189:45453)
0810 18:53:00.314218 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:00.314222 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:00.314688 (+ 466us) server_negotiation.cc:407] Connection header received
0810 18:53:00.314895 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:00.314898 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:00.314968 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:00.315032 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:00.315503 (+ 471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51716: BlockingRecv error: recv got EOF from 127.0.0.1:51716 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260810 18:53:00.320430 12786 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:53:00.319012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51722 (local address 127.18.168.189:45453)
0810 18:53:00.319106 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:00.319110 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:00.319608 (+ 498us) server_negotiation.cc:407] Connection header received
0810 18:53:00.319780 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:00.319783 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:00.319829 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:00.319877 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:00.320342 (+ 465us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51722: BlockingRecv error: recv got EOF from 127.0.0.1:51722 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260810 18:53:00.326185 12786 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:53:00.324619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51730 (local address 127.18.168.189:45453)
0810 18:53:00.324794 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:00.324799 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:00.325306 (+ 507us) server_negotiation.cc:407] Connection header received
0810 18:53:00.325492 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:00.325496 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:00.325547 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:00.325597 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:00.326087 (+ 490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51730: BlockingRecv error: recv got EOF from 127.0.0.1:51730 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260810 18:53:00.333936 12786 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:53:00.332342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51734 (local address 127.18.168.189:45453)
0810 18:53:00.332478 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:00.332483 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:00.333019 (+ 536us) server_negotiation.cc:407] Connection header received
0810 18:53:00.333213 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:00.333216 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:00.333266 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:00.333315 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:00.333821 (+ 506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51734: BlockingRecv error: recv got EOF from 127.0.0.1:51734 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260810 18:53:00.346271 12786 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:53:00.344403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51746 (local address 127.18.168.189:45453)
0810 18:53:00.344523 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:00.344528 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:00.345125 (+ 597us) server_negotiation.cc:407] Connection header received
0810 18:53:00.345343 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:00.345347 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:00.345423 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:00.345477 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:00.346109 (+ 632us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51746: BlockingRecv error: recv got EOF from 127.0.0.1:51746 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260810 18:53:00.367197 12786 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:53:00.365196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51750 (local address 127.18.168.189:45453)
0810 18:53:00.365328 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:00.365333 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:00.365957 (+ 624us) server_negotiation.cc:407] Connection header received
0810 18:53:00.366221 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:00.366225 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:00.366283 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:00.366365 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:00.367036 (+ 671us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51750: BlockingRecv error: recv got EOF from 127.0.0.1:51750 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260810 18:53:00.403375 12786 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:53:00.402032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51764 (local address 127.18.168.189:45453)
0810 18:53:00.402196 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:00.402201 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:00.402396 (+ 195us) server_negotiation.cc:407] Connection header received
0810 18:53:00.402570 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:00.402574 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:00.402628 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:00.402725 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:00.403223 (+ 498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51764: BlockingRecv error: recv got EOF from 127.0.0.1:51764 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260810 18:53:00.470938 12786 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:53:00.469576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51774 (local address 127.18.168.189:45453)
0810 18:53:00.469759 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:00.469764 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:00.469950 (+ 186us) server_negotiation.cc:407] Connection header received
0810 18:53:00.470163 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:00.470167 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:00.470227 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:00.470319 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:00.470823 (+ 504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51774: BlockingRecv error: recv got EOF from 127.0.0.1:51774 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260810 18:53:00.602473 12786 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:53:00.601393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51790 (local address 127.18.168.189:45453)
0810 18:53:00.601564 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:00.601568 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:00.601637 (+ 69us) server_negotiation.cc:407] Connection header received
0810 18:53:00.601768 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:00.601770 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:00.601819 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:00.601894 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:00.602348 (+ 454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51790: BlockingRecv error: recv got EOF from 127.0.0.1:51790 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260810 18:53:00.861930 12786 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:53:00.860710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51802 (local address 127.18.168.189:45453)
0810 18:53:00.860874 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:00.860880 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:00.860923 (+ 43us) server_negotiation.cc:407] Connection header received
0810 18:53:00.861126 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:00.861128 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:00.861179 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:00.861247 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:00.861794 (+ 547us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51802: BlockingRecv error: recv got EOF from 127.0.0.1:51802 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 18:53:01.175967 12898 catalog_manager.cc:1259] Loaded cluster ID: 79112864ed444994b387422e3a593cc5
I20260810 18:53:01.176076 12898 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d684a82e5f7f43eeb3864a55d96c0418: loading cluster ID for follower catalog manager: success
I20260810 18:53:01.177826 12898 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d684a82e5f7f43eeb3864a55d96c0418: acquiring CA information for follower catalog manager: success
I20260810 18:53:01.178980 12898 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d684a82e5f7f43eeb3864a55d96c0418: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 18:53:01.180576 12912 catalog_manager.cc:1259] Loaded cluster ID: 79112864ed444994b387422e3a593cc5
I20260810 18:53:01.180642 12912 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 22bd4a7709224b68a73df6406aa7c8df: loading cluster ID for follower catalog manager: success
I20260810 18:53:01.182312 12912 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 22bd4a7709224b68a73df6406aa7c8df: acquiring CA information for follower catalog manager: success
I20260810 18:53:01.183405 12912 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 22bd4a7709224b68a73df6406aa7c8df: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 18:53:01.379920 13009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:01.376053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51816 (local address 127.18.168.189:45453)
0810 18:53:01.376350 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:01.376353 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:01.376369 (+ 16us) server_negotiation.cc:407] Connection header received
0810 18:53:01.376503 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:01.376506 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:01.376641 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:01.376760 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:53:01.377025 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:01.377570 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:01.378403 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:01.378623 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:01.378695 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:01.378986 (+ 291us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:53:01.379482 (+ 496us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:53:01.379597 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:01.379660 (+ 63us) server_negotiation.cc:299] Negotiation successful
0810 18:53:01.379710 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":81,"threads_started":1}
I20260810 18:53:01.386706 13015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:01.383188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38722 (local address 127.18.168.188:34063)
0810 18:53:01.383427 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:01.383431 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:01.383443 (+ 12us) server_negotiation.cc:407] Connection header received
0810 18:53:01.383522 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:01.383524 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:01.383641 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:01.383734 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:53:01.383930 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:01.384421 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:01.385278 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:01.385502 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:01.385617 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:01.385852 (+ 235us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:53:01.386240 (+ 388us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:53:01.386326 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:01.386454 (+ 128us) server_negotiation.cc:299] Negotiation successful
0810 18:53:01.386505 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":97,"threads_started":1}
I20260810 18:53:01.393941 13009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:01.389615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51832 (local address 127.18.168.189:45453)
0810 18:53:01.389743 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:01.389747 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:01.390213 (+ 466us) server_negotiation.cc:407] Connection header received
0810 18:53:01.390391 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:01.390395 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:01.390449 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:01.390522 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:53:01.390881 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:01.391691 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:01.392805 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:01.393110 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:01.393209 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:01.393431 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:53:01.393519 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:53:01.393590 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:01.393733 (+ 143us) server_negotiation.cc:299] Negotiation successful
0810 18:53:01.393801 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 18:53:01.394433 13015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:01.389695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38732 (local address 127.18.168.188:34063)
0810 18:53:01.389795 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:01.389802 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:01.389994 (+ 192us) server_negotiation.cc:407] Connection header received
0810 18:53:01.390243 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:01.390246 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:01.390297 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:01.390346 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:53:01.390686 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:01.391362 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:01.392481 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:01.392863 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:01.392963 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:01.393837 (+ 874us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:53:01.393912 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:53:01.393961 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:01.394272 (+ 311us) server_negotiation.cc:299] Negotiation successful
0810 18:53:01.394325 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 18:53:01.394659 13021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:01.389571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46436 (local address 127.18.168.190:36165)
0810 18:53:01.390283 (+ 712us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:01.390289 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:01.390304 (+ 15us) server_negotiation.cc:407] Connection header received
0810 18:53:01.390697 (+ 393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:01.390701 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:01.390846 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:01.390964 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:53:01.391164 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:01.391903 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:01.392963 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:01.393320 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:01.393438 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:01.393736 (+ 298us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:53:01.394049 (+ 313us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:53:01.394153 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:01.394290 (+ 137us) server_negotiation.cc:299] Negotiation successful
0810 18:53:01.394350 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":92,"threads_started":1}
I20260810 18:53:01.397656 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 12625
I20260810 18:53:01.407362 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 12707
I20260810 18:53:01.417645 19106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1VMCzb/build/debug/bin/kudu with pid 12789
2026-08-10T18:53:01Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3154 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6314 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (293554 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST